Let's Make Robots!

"Home Robotics" musings - what needs to be done

Hey guys. I wrote this two years ago before creating my robot. Obviously my vision has a long way to go. But I felt then and now that the biggest issue I could help overcome was the mobility side of the equation. Check this out and tell me where I'm wrong! I won't be happy until there is a robot in every home! :o)

 

Arti – Articulated Mobile Reconnaissance Platform

Helpful – practical – mobile – affordable – friendly – easy to use

The state of the home robotics industry has been one of chaotic disappointment. Moore’s law – which promises the doubling of processing speed at half the price every eighteen months – has long been argued to usher in a new era of home robotics. The science fiction promise of a robot companion or worker in every home has long been the force both pulling and pushing innovation. However, the realities have been far from resembling the promise. The reasons for this repeated disappointment are primarily two fold.

First, although the prescience of Moore’s law has certainly proven true and has every expectation to be fulfilled for at least the next decade, it applies to hardware, and not sophisticated software logic and code. Simply stated, it has been much harder than anticipated to develop the operating systems needed to run the artificial intelligence required for a valuable home robot.

Second, by the nature of the purists who are most interested in robotics – science fiction fans, technically oriented hobbyists, and academicians – the efforts and goals have been too far reaching and the results have been too complex and difficult to use. There has been an over reliance or desire for humanoid forms. There has been an assumption of technical prowess of the owner. There has been an unrealistic desire to have all of the processing capabilities fully onboard the robot.

However, there has been some success: Aldebaran’s Nao, iRobot’s Romba vacuum cleaner and their most recent prototype Ava, healthcare specific InTouch Technologies (primarily telepresence), KIVA systems (warehouse robots), Mobile Robots, Inc., Yujin iRobi, Whitebox Robotics, RoboDynamics’ Luna, and the creations from Honda and Sony.

But all of these players are missing some or many critical functionalities in order for their robots to be suitable for home companion use. A successful home companion robot should be helpful, practical, mobile, affordable, friendly, and easy to use.

Helpful: it needs to actually do something for the owner that provides immediate value

Practical: it needs to be available, always on, accessible, etc.

Mobile: it needs to be able to traverse the building in an easy and timely manner

Affordable: it needs to be priced between the cost of a nice PC and a car

Friendly: it needs to have a strong “conversational AI” for fun and enjoyment

Easy to use: no-hassle guarantee. It works even for non-technical folks upon delivery

The key “must haves” for home companion robots are:

  1.  Fully set up and supported by technical professionals – technical skills must not be needed to own and operate
  2. Full home and timely mobility (must be able to easily climb variable stairs, self aware location, spatial mapping, obstacle avoidance, ability to follow owner, etc.)
  3. Full access to and utilization of local network and the Internet to “offload” high level processing needs
  4. Telepresence – ability to communicate with others (automated cell phone dialing, Skype, email, texting, etc.)
  5. External control option so loved ones, physicians, care takers, etc. can easily communicate with owner
  6. Intelligent verbal and visual (voice and screen) responses
  7. RealVoice™ conversational tone and language set
  8. RealFace™ avatar on high definition screen customizable for the owner – friendly and familiar
  9. Ability to understand data requests and perform information searches
  10. Simple tasks like remote television controls
  11. Touch screen interface
  12. Fully encrypted communications
  13. Self charging (with inductive pad)

Additional “nice to have” features include:

  1. One or two arms and hands for grasping and carrying objects
  2. Artificial Intelligence (AI) that provides for reasonable ability to “chat” with owner on diverse and unplanned topics
  3. Built in and learnable Visual Recognition Libraries (VRL’s: e.g. identifying chair, couch, door, pets, etc.)
  4. Emergency care taker monitoring and communicating – multiple levels of help (technical support, family, physician, 911, etc.)
  5. Home controls – heat, security, lights, etc.
  6. Training for simple physical tasks with “arm and hand” (open door, pick up newspaper, etc.)
  7. Search and communicate breaking news
  8. Live television / movies on screen
  9. Tracking of conversations with “key word ranking” to learn owner’s likes and to be more conversational
  10. Tracking of phone calls and other communications (you haven’t called Andrew for ten days)
  11. Built in games (using standard apps, plus dancing, other physical activities)
  12. Appointment schedule reminders, etc.
  13. RealFace™ to include eye and “head” (monitor) tracking
  14. Wireless Printing
 

 

 To be completed . . .

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Thanks for putting your thoughts out there, got me thinking.  I always appreciate anyone that does that.

In order to be commercially successful, robots must satisfy a human need (obvious), as I don’t see a robot able to fix a leaky faucet anytime soon, I think the human need for interaction, companionship, and personal information agent is more realistic.  You mention the elderly, which I think is one very large market.   I’ve been talking to a lot of parents lately that are interested in robots for the young, to read to them, act as tutors, companions, pets, monitors, etc.  I think this market would be achievable in the not too distant future.  I've been assembling some use cases.  I think the young would put up with a lot of deficiencies that the old won’t as the technologies improve and mature.  So, young and old, in between you have bachelorhood…I would want a cool robot as a bachelor…and married life…what mancave doesn’t need a good robot.  Bartending skills would be awesome.  In order for a person to bond with a robot enough to not turn it off or leave it uncharged after the first week, I have recently been re-thinking my priorities towards that goal.  A few months ago I didn't see the value in half the stuff I'm about to mention.

Having said all that, here are my must haves as I see them.

1.       1.  Robots must have a face with moving features capable of showing all kinds of facial expressions.

2.       2.  Robots must have an emotional engine that mimics the state of the art in our modeling of emotions and mood.

3.       3.  Robots must have a personality at a point in time, however, there is no reason why new personalities can’t be loaded on the fly, downloaded, tried out, and customized to each users preferences…which means robots will probably have multiple personalities in a given day…When the wife isn’t home or the kids go to bed, a different personality could come on.  In my opinion, a personality could be represented by a set of integer settings…each setting being something like “Likes to Talk”, “Likes to Make Jokes”, “Likes to Be Useful”, “Likes to Move”, “Likes to do what humans ask.”  I think a schizophrenic bot could be much more interesting to have around.  I could see app stores of new downloadable personalities that people would try out until they find one or a few they really connect with.

4.       4.  Better Listening Libraries…hopefully self-contained.

5.       5.  Better Speech Libraries…Almost everything that a robot says should be impacted (and said differently) depending on the emotional state (Happy, Angry, Sad, Fearful, etc) of the robot, the degree of that state (0-100%), and one or more personality attributes (Politeness, Morality, Humor, etc)  New libraries / speech engines / abstraction layers are needed to make programming listening and speech much easier to program.  I think the programmer should decide on the concept to be spoken, and then let the engine decide how to express the concept in a given language, emotional context, politeness level, etc.  One verbal concept could result in hundreds or thousands of variations.  This is necessary to prevent monotony.

6.       6.  The Robot should decide what to do based on its personality and environmental situations as they happen.  For a robot to be bonded with, I don’t think it can just sit around waiting for a human to give it a specific order.  It’s quite boring.  In order to accomplish this, the bot must be able to poll all its personality attributes (or something else), evaluate priorities, and DECIDE on a course of action or inaction.  The variation can and should be infinite.  Diverse autonomous behavior will just flow out of this type of model, rather than needing to be planned out and programmed procedurally ahead of time...which I think is impractical for rich behavior.  

7.       7.  3D Vision Perception and Memory.  Robot needs to perceive and remember everything it sees in 3 dimensions, or at least the latest state and what the state was yesterday, last week, last year, etc…as people do.  The robot should have an answer for the basic question “Where was the last place you saw my keys?”  I think the software libraries are so very far from being anywhere close to having the right structure and ideas to make this happen.

8.       8.  Localization, obviously.  Robot must know where it is in real-time and from a coldstart…quickly.  Once again, the software isn’t really there yet in my opinion...not remotely close.

9.       9.  Mobility.  I differ with you somewhat here.  I love your stair climber.  Love it.  I differ in that I don’t believe it’s the primary challenge.  I started programming software 30 yrs ago, and I still am not impressed with how it’s progressed (Watson is a notable exception).  I think numbers 2-8 on my list are bigger challenges, especially the language and vision stuff.   I only say this because I see software as the lacking element that is still so very far from being good enough, software good enough for a person to feel something for their bots.   Someone will invent a lot of good platforms eventually, but someone has to have a reason to want to keep paying the power bill to keep them charged.   I think wheels or tracks would be fine for awhile.   Most people would carry their bot up and down the stairs if it asked nicely and they had a connection of some kind with the bot…like you would a pet.

10.   10.  Better and Cheaper Perceptual Sensors, or pre-packaged sensor arrays that can more easily be added “Mr. potatohead” style to a bot.

11.   11.  Networked, Internet Access, Phone Access, Text Messaging, etc.  I think all that is so basic it almost goes without saying.  I think robots could also participate in social media as long as privacy can be controlled by the humans that co-habitate.

12.   12.  Standards, Standard, Standards…doubt it will happen in my lifetime.  Better Plug-n-play standards for Sensors/ Actuators, and Software Interfaces for various bot services, etc. would sure speed the pace of innovation.  In the future, I think software developers should concentrate on building a single new personality attribute, behavior, or service that can then be published to an app store and downloaded into many different kinds of robots running modules downloaded from all kinds of sources.  This would make robots evolve…quickly.  The task of building a beloved robot will be too great and take too long to happen in my lifetime without something of this nature to allow building behaviors in smaller decoupled pieces.  This is near and dear to me as I used to build reusable software frameworks for a living.  In other domains, the number of lines of code can be cut down 10x-1000x with the right ideas.  I hope this too can happen for robot behavior.

 I’m I'm intentionally leaving out any mention of arms, hands, etc.  I think it might be best to let the above capabilities develop and mature with adequate safety mechanisms in place before giving robots opposable thumbs and letting some alternate personality decide to teach your kids to clean your guns for you while you’re away.  Opposable thumbs mean using doorknobs, keys, cars, gas ovens, a lotta stuff I’m not comfortable with in this decade.  Of course, innovators such as us should go on doing what we do, thumbs and all!

 

Obviously, a lot of subjective opinion here, hope it inspires some thought.  Whether you agree or disagree with anything I’ve said, I applaud all those that have the inspiration to have an idea and the persistence to see it through.  Happy botmaking!   

Regards,

Martin

Hey Bill. Great comments. Truly. I'm walking out the door for an 8 day motorcycle vacation so I can't respond in detail, but I intend too. I think our visions are fairly well aligned - other than the kids. I'll have to think about that one. I shudder at the idea of parents not reading to kids and having robots do it instead. But that may be my age - and personal preference showing. I've had few joys greater than having my children sit on my lap as I read to them. But I'm sure you noted my "RealVoice" and "RealFace" mentions. I've separately written up specs on those that are aligned with you suggestions. I couldn't agree more. I also agree on the personality, etc. I envision a bot that pays attention to what their owner reads online, etc. and then searches for current information on those topics and then "converses" with the owner pertinently and in timely and informative manner. I think in making our lists, it's imperative to know what is possible today (next ten years) and not likely and to stay away from promising the "not likely." But especially in the elder care market, if we can overcome the fear factor of having a robot in the house (that's why the face and voice and personality are so important) there is so much we can do right now to help that older generation. Imagine their robot being their eyes, ears, voice and conduit to the world. The voice recognition is crucial. As you mention, how great if it were "resident" rather than cloud based. But Google has certainly proven the speed and efficiency of cloud based VR.

 

Anyway. I'll try to write back more thoroughly in 9 days! Thanks so much for the response and discussion. What a fun, bright, and intellectually stimulating area of vision!

What is interesting about your list, is that all but a few of them are available now and almost all of them are software solutions.  Ok, the ones that aren't there are HUGE problems to solve.  Yet your invention in a step toward that goal (cool as it is and it is pretty cool!) was a hardware solution.  I write software for a living, and I look at it and see that the work toward your goal is really a software integration problem, not hardware.  But, if you have a hammer in your hand, you are more prone to hit the nail with that hammer than the one at your tool belt.  I am interested in your perspective and why you went that direction is all.

Dean Kamen (Segway inventor) built a wheelchair for going up and down stairs several years back.  I think they tried to get FDA approval and never were able to even with huge resources etc going into it.  I saw some videos of it at the time and was very impressed with what his team did in terms of stability etc. so not sure why it didn't make it through FDA approval.  I mention it since it was a very different approach which was pretty successful (not successful enough for the FDA though) and might be cheaper to manufacture than the number of moving parts on your existing design. 

I also agree with Max's suggestion that the market will start as niche oriented specialized gadgets since the price to integrate existing technology will remain high for quite some time.  

Thanks for getting me thinking.  Good luck with your invention.  It is pretty nifty..

Regards,

 

Bill

I totally agree with your perspective, Bill. The funny thing is that I have a software background but kept seeing what I considered failures in the market from a locomotion / hardware standpoint. It seemed to me, perhaps wrongly, that the biggest problem was one of maneuverability. it's kind of a chicken egg issue. No matter how great the software solutions - and some are getting pretty darn close - if they are stuck on a non-autonomous non-mobile computer then they are little more than apps. When I saw the many million dollar Honda robot fall over trying to climb stairs made specifically for it and for which it had been calibrated for half a day, I said to myself that a lower tech solution was needed. Kamen's wheelchair, by the way, is magnitudes more complex (and cooler) than my solution. And at $25k each, way too expensive for "home" robotics. I have no interest in pursuing moving people up and down stairs. Just robots / telepresence. He did (Johnson and Johnson), by the way, get FDA approval. That wasn't the problem. But it was a very complex machine (complex things tend to break a lot) and very expensive. It also weighs around 300 pounds without a human in it. 300 pounds (or even 30 pounds) is a non-starter from my perspective. My apparatus is weighing in around 15 pounds.

So, it's good to disagree because it can lead to deeper thought. For me, my "angle of attack" is from the telepresence / senior companion standpoint for home robotics. As baby boomers age, they are going to need "devices" that can extend their ability to stay in their homes and keep their independence. Combine this with a geographically dispersed family, I can see a big market for telepresence "robots" that  assist with memory and information and companionship and yes, small tasks as well. But the latter, the tasks, are the last in the list of what will sell - methinks.  

I didn't know they got the FDA approval.  I live 10 miles from Deka so Kamen breathes, and it is in the press here.  I hadn't heard anything about it since he inked the deal with J&J hence my assumption.  It definitely is a complex system.  My thoughts were that a simplified version might be easier to manufacture and/or give something that can have a higher cg than what arti could support. 

I just read on yahoo my first stock valuation of robot related companies. 

 http://beta.fool.com/bluemarkanalysis/2013/07/13/the-robots-are-coming/40200/?source=eogyholnk0000001 

Someone is going to make some serious money with robots.  You are right that tasks might not be the market to be in since they are very down on iRobot.  IRobot is competing with people with brooms and many knock off competitors who have leaped into the market after them in what is an overcrowded market with some #300 gorillas, Whirlpool etc.

For a senior companion, maybe instead of a robot that wanders behind you, sell or rent modules with video, speaker and mic in each room where someone might want it.   It can sense when someone gets up in the morning and then via voice reminds about doctor's appointments upcoming birthdays etc.  This could be marketed as security upgrade along with an easy way for families to connect via skype or whatever and a way to keep an eye on someone.  Even if someone falls in their bathroom without one of these devices in the bathroom, the system will know they haven't moved out of the bathroom for several hours and can call for help.

This way you aren't constrained by cpu, battery, the hard problem of going up and down stairs etc.  This could be a standard PC solution using mostly off the shelf products with integration work to make it complete.  The only embedded work is the modules streaming encrypted video to a main server wirelessly.  The hardware cost would be more than using Arti as a base, but that would be well offset by not having to write as much custom software.  Sell it as a service like a cell phone and the consumer only sees it as a relatively low monthly cost.  If it can keep someone in their home longer, that cost would be negligible. 

There are pros and cons to both.  You seem a smart guy and probably has already thought this through more than I have and discarded it.  I look forward to seeing a little Arti running around in my folks house in a few years keeping an eye on my folks.

Good luck to you.

Regards,

Bill

 

 

 

 

 

Yes, the "smart home" vision has a great amount of potential - much like the everpresent computer in Star Trek and the movie IRobot. But my feeling is that the "Big Brother" sense of always being monitored (yes, easy to mute, etc.) will keep that from the mainstream for a long time. Also, the costs of wiring an entire house (obviously house dependent) could be prohibitive - even wireless requires a unit in each room. But you are 100% correct that many of the functions / features I wrote about would do well in that manner. However (there are always alternatives) the problem with a smart home is that it can't do physcal tasks at all. So an Arti based (or some other moving base) robot "can" do tasks it's just a matter of programming what is needed and having the hardward to accomplish it. But tasks do require a moving platform. Also, just as people love pets, I'm thinking a robot companion could take on many of the same roles as pets . . . and then flip into a person via a Skype like call, etc. I can imagine, like on the Xbox online platform, a nearly infinite number of personalities and personalizations to make your bot yours.

Without looking, I'm sure IRobot is down in value due to the winding down of the two wars. I'm guessing their mililary bots were cash cows (but I'm just guessing.) Thanks for sharing your thoughts!

The problem is that a domestic robot that can perform useful functions depends on a large number of different technologies working together. With current technology such a robot would cost far more than a car. 

Even if we start with something simple such as mobility we soon run into all sorts of technical problems. Lets use your "Arti" chassis as an example.

As a chassis without any load it handles stairs and flat ground very well. Once you add a humanoid torso that weights 50Kg or more then it will fail even if you beef up the motors, suspension etc.

Arti's long, snake like chassis is not really suitable for a torso. The weight distribution will change completely and the robot will tend to fall over sideways or backwards.

Good points which kind of reflect my opinion that a humanoid approach is way in the future and we need to be focusing on utiilitarian / high value ROI's. I believe the "elder care" market, which is tangential to telepresence, is ripe for a robot companion able to easily navigate an entire house / home / apartment, etc. I agree that my Arti design definitely needs a low center of gravity. No tumbling allowed front, back, or sideways! :o)

It seems like you're criticizing the fact that people have highly specific and unrealistic Sci-Fi expectations for home robots but then demanding highly unrealistic Sci-fi features in the end.

I can't prophecy here, but my guess is that home robots won't be one thing for all purposes, rather several simpler semi-intelligent objects doing useful but less complicated tasks for us. It may not make sense for your laundry folder to be the same machine as your cook. (Bad example-they'd probably have about the same dexterity and complexity, but I think you get the point.). You don't use a Roomba to clean the pool (not if you're wise anyway) but the Roomba is still pretty useful. The future might look more like a "house of tomorrow" than Sonny from "I, Robot."

Which of my suggestions do you think are unrealistic sci-fi features? If I included them, then I thought they were reasonable over the next decade. But I most certainly could be as wrong as prognosticators of the past!

I'm hoping for / betting on a mobile companion / telepresence robot who can "talk" to you, answer your questions (like an improved siri) and perform certain easy tasks, not the least of which is provide you (think elderly) with a powerful (and mobile) link to the outside world. Maybe I'm just dreaming. :o)