So, Where’s My Robot?

Thoughts on Social Machine Learning

NYT on Robot Companions

Latest in the “Smarter than you think” series. The NYT ran an article yesterday on personal robots.

Paro as emotional companion in the nursing home, and Autom as personal weight loss coach are the most prominently featured examples in this article that explores the touchy subject of robot companions, and what kind of relationships we might let robots hold in our lives.

July 6th, 2010 Posted by | HRI, In the News | no comments

Mr President, Let’s not call it AI

I had a unique opportunity yesterday, I was invited to participate in a PCAST workshop (the President’s Council of Advisors on Science and Technology).   The theme of the meeting was Bio/Info/Nano tech, what exciting opportunities are happening in these fields that will create jobs in the US, and what the government can do to spur innovation.   I’ll probably have a couple of SWMR posts about the discussion, and thought I’d start off with one about my contribution to the discussion, since I was the only roboticist in the room.

They had a wide range of discussants, several of us were early career researchers, which I think were invited to share our “what’s new and exciting that’s going to create jobs” point of view.  Another contingent of the discussion group were more seasoned researchers and entrepreneurs, that had a sort of from the trenches perspective of how the government’s support of basic research has changed over the years.

Each discussant had the opportunity in a three minute introduction to make a statement to the council.  Here’s a recap of what I said:

The technology opportunity I decided to highlight is service robotics, because they have the potential to dramatically impact such a diverse set of societal needs.  Robots that are capable of working alongside people will revolutionize workplaces, for example in manufacturing.

Robotics represents perhaps our best opportunity to achieve higher levels of domestic manufacturing agility and overall productivity needed to retain high-value manufacturing jobs in the U.S., provided that the current state of the technology can be significantly advanced.

Today’s industrial robots lack the capabilities required to do more than just blindly execute pre-programmed instructions in structured environments.   This makes them expensive to deploy and unsafe for people to work alongside.

There is an opportunity to usher in a new era of agile and innovative manufacturing by developing service robots as co-workers in the manufacturing domain.  These capable assistants would work safely in collaboration and close proximity to highly skilled workers.   For example, providing logistical support, automatically fetching parts, packing/unpacking, loading, stacking boxes, emptying bins, detecting and cleaning spills.

Very similar logistical robotic support could help streamline the operation of hospitals, driving healthcare costs down.

In order to realize this vision, we need to move beyond robots only operating in relatively static structured environments.  This presents several research challenges, and I think that the following three are most critical to progress.

– This requires advances in sensing and perception technology, allowing robots to keep track of a dynamically changing workplace.

– Manipulation is a key challenge as well, robots need the flexibility to be able to pickup and use objects in the environment without tedious pre-programming of specialized skills.

– Finally, an important challenge in bringing these robots to fruition is advances in human-robot interaction.  We need these robots to work safely and efficiently in collaboration with human workers.  People can’t just be seen as an obstacle for the robot to navigate around, the robot needs to reason about and understand people as interaction partners.

Recently, over 140 robotics experts across the country have come together to articulate a national robotics initiative, a robotics research roadmap.  This roadmap lays out the target areas where we think robotics research efforts need to be supported in order to bring about robot technology that will have the biggest impact on our economy and our society.

The comment I got from one of the council members was interesting, she said (I’m paraphrasing) “Aren’t you leaving out the challenge of Sentience or AI needed?”  I only had time for a short answer, and said something to the effect that, yes, I think that the notion of AI cuts across all of the three areas I mentioned, but particularly human-robot interaction.  In order for a robot to work side-by-side with a human partner it will need human compatible intelligence capabilities.

But here on SWMR, I’ll give the longer answer….that, no I don’t think we need AI for service robots.  Or I don’t think that’s what we should call it.  Yes, perception and manipulation and HRI and autonomy in general all fit under the big umbrella term of AI.  But the term AI is so vague, and it makes people think of science fiction, which then makes you feel like robots in society is some pipe dream far in the future.  So, particularly in settings like PCAST where people want to hear about concrete objectives and job creation, it does our field no good to just lump everything under the term AI.

If instead we talk about the specific intelligence challenges suddenly it all seems much more achievable, and you can imagine some semi-autonomous form of service robots being deployed in the not so distant future.   We see that, hey sensing technology is getting better and better, and look at all the academic and industrial partners working on the manipulation problem, that seems achievable.  And in terms of AI for human-robot interaction, yes we need to make some significant advances in computational models of social intelligence before robots can truly interact with people in unstructured environments.  But do we need to solve AI?  I don’t think so.

June 23rd, 2010 Posted by | Conferences, HRI, Industry | no comments

Telepresence Robots

There’s been a recent flurry of announcements of telepresence robots.  QB is now available from the CA based Anybots (shipping Fall 2010).  Texai is the WillowGarage platform (not yet for sale).  And Vgo from startup, Vgo Communications, was recently covered in the Boston Globe.

Some visions of these platforms include: telecommuters logging in to interact with their co-workers at the office, or attending a meeting with co-workers in another city, or using it to check out a situation at the factory from the comfort of your office in another location.  While you’re probably not going to buy one for personal use, (QB has a $15K price tag, and Vgo is about $6K) you might just start to see them roaming around your office.

Vgo and Texai look more like laptops on wheels, but QB has a more anthropomorphic presence that I think is nice.  It’s adjustable height lets it get fairly tall, which is necessary for the telecommuter to get any reasonable view of the remote location, and allows it to participate in standing or sitting conversations.   Yet it is small/skinny/light which makes it safe and easy to operate in close proximity to people.   You drive it with the arrow keys on your keyboard, to make it go L,R,forward,back.

One thing I think it missing from all of these telepresence robots, but particularly from an anthropomorphic one like QB is a neck tilt.   For utility purposes, I imagine it would be nice to scan up and down with the camera.  But also for social purposes, it would be nice for the person to trigger head nods (and shakes if we could add another DOF).   If this robot is going to be standing in for people in conversations, it could be awkward if it doesn’t display any backchannel communication.  People will get used to the robot not doing it, but it’s presence in the conversation would be much stronger with a backchannel.  These are the little “yeahs” and “uh-huhs” that people mutter to let the speaker know “I’m with you, keep going.”   Much of this happens over the speech channel, which QB will capture, but we also use body language to communicate this back channel info to the speaker.   Allowing QB to nod would let the telecommuter give both physical and verbal backchannel to the person they are speaking with.

I’m excited to see how people start using their Anybots and Vgos, the water cooler will never be the same!   Here’s a video of QB in action.

June 9th, 2010 Posted by | HRI, Industry | 3 comments

PR2 Graduation + GT Spotlight

Nice to see our GT team highlighted on the Willow Garage page today:

Robots like the PR2 may be able to help older adults stay in their homes longer with a high quality of life. The Georgia Tech team aims to make progress towards this long-held dream.  Rather than try to guess what seniors want, the team will work with older adults to better understand their needs and how robots can help. The team will also write code to make the PR2 perform helpful tasks at home. By working closely with seniors throughout the research process, the team hopes to better meet real needs and accelerate progress. To make everything more realistic, the robot will spend some of its time in a real, two-story house on the Georgia Tech campus, called the Aware Home.

They will be doing a spotlight for each of the eleven PR2s heading off to research labs this summer.   The robots were sent off with quite the fanfare, this video (via IEEE Spectrum) captures the “graduation” event nicely, including a brief interview with someone from each team.  The group as a whole is tackling a wide variety of personal robotics challenge problems!

June 7th, 2010 Posted by | GT Lab Updates, HRI, Industry | no comments


FloBi is a cute and nicely designed social robot head from University of Bielefeld in Germany (presented this week at ICRA 2010).  It is specifically designed for HRI and is anthropomorphic but has a cartoon-like baby face to get around the uncanny valley.  Which I think they definitely achieved.

One of the most interesting aspects is the modular design.  As seen in the video linked above, the shells are easily changed out to create different looks, changing from male to female for example.  Another unique characteristic is the facial features.  They had the goal of having “no holes” in the face, which lead to a magnetic actuation design for the facial features (lips,brows).  This was fun to see working, since its an idea we have been playing around with independently.

Looking forward to seeing more about what comes from the FloBi project!

May 6th, 2010 Posted by | Conferences, HRI | no comments

PR2 is headed to Georgia Tech

Today Willow Garage announced the winners of their PR2 Beta Program CFP.   After reviewing 78 proposals they selected 11 schools to receive a PR2, and our Georgia Tech team, headed up by Prof. Charlie Kemp, made the cut.

  • Albert-Ludwigs-Universität Freiburg with the proposal TidyUpRobot
  • Bosch with the proposal Developing the Personal Robotics Market.
  • Georgia Institute of Technology with the proposal Assistive Mobile Manipulation for Older Adults at Home.
  • Katholieke Universiteit Leuven with the proposal Unified Framework for Task Specification, Control and Coordination for Mobile Manipulation.
  • MIT CSAIL with the proposal Mobile Manipulation in Human-Centered Environments.
  • Stanford University with the proposal STAIR on PR2.
  • Technische Universität München with the proposal CRAM: Cognitive Robot Abstract Machine.
  • University of California, Berkeley with the proposal PR2 Beta Program: A Platform for Personal Robotics.
  • University of Pennsylvania with the proposal PR2GRASP: From Perception and Reasoning to Grasping
  • University of Southern California with the proposal Persistent and Persuasive Personal Robots (P^3R): Towards Networked, Mobile, Assistive Robotics
  • University of Tokyo, Jouhou System Kougaku (JSK) Laboratory with the proposal Autonomous Motion Planning for Daily Tasks in Human Environments using Collaborating Robots

May 4th, 2010 Posted by | Announcements, HRI, Industry | no comments

Robot Jazz Improv

CNN ran a story last week on the latest work of Gil Weinberg and Guy Hoffman on their robot musician, Shimon.   The story covers their recent demonstration of robot jazz improv at CHI 2010 and the Listening Machines 2010 concert.

“Shimon is an autonomous marimba-playing robot designed to create interactions with human players that lead to novel musical outcomes. The robot combines music perception, interaction, and improvisation with the capacity to produce melodic and harmonic acoustic responses through choreographic gestures. We developed an anticipatory action framework, and a gesture-based behavior system, allowing the robot to play improvised Jazz with humans in synchrony, fluently, and without delay. In addition, we built an expressive non-humanoid head for musical social communication.”

The new head is particularly nice–a 6-degree of freedom addition, that aesthetically matches the industrial looking mallet arms.   It uses the camera in the head to perceive other band members, and times its head beats to the music.  As they say, the purpose of the head is to create the social responsiveness and reaction that the other humans in the jazz group expect from a teammate.

And I’m not the only one who thinks they did a nice job on the social cues, the local Atlanta news anchors liked it too (though they did list the article under the heading “bizarre”).

Update:  This work just received a best paper award at ICRA 2010, congrats to Guy and Gil!

May 2nd, 2010 Posted by | HRI, In the News | no comments


We are excited about how well Simon did at the CHI 2010 Interactive Demo session last week.  Our demo got a lot of traffic, especially during the opening event on Monday evening, and even got some coverage on PC World (who did the video below), engadget, and NPR.

This was Simon’s first venture out of the lab, so it has been interesting for forcing us to do more on-board perception, and generally putting the platform through its paces.  We were doing an interactive learning demo, using active learning, where Simon learns a “cleaning up” task.  The human teacher provides examples of what goes where.   Simon uses these examples to build a model of what goes in each location.  The teacher can also ask Simon if he has any questions, and the robot will point to or pick up an object that it is least certain about where it goes.  In addition to learning, we added perception capabilities to give Simon more ambient awareness of the environment, with an attention mechanisms that pays attention to visually salient cues as determined by the Itti saliency model, as well as faces, and loud sounds.

Simon got to interact with hundreds of people, and was particularly popular with kids of the conference attendees.

And also got to meet the designer of his head shells, Carla Diana, who finally got to see the finished product in person.

April 18th, 2010 Posted by | Conferences, GT Lab Updates, HCI, HRI | no comments


I recently taught a two-week introduction to HRI for our new crop of Robotics PhD students.   It was an interesting exercise, to boil down some of the main topics and issues of HRI into six lectures/meetings.   My goal was to communicate how some traditional robotics problems become conceptually different when you add a human-in-the-loop. And that there are some fundamental human abilities that make up social intelligence.

One amazing human capability is our propensity to recognize goal-directed motion. Whereas a computer vision algorithm struggles to parse a stream of video and determine whether there are people in it and what they might be doing, humans do this naturally from a very young age.

I think one of the most interesting findings in this realm for a roboticist is the work with infants that shows some of the principles of recognizing goal-directed action. Csibra performed a series of experiments with infants and children looking at the how they interpret simplified intentional action represented by geometric shapes, something like this famous Heider-Simmel video.

People watch this video and see a complex series of social dynamics and goal-directed actions. Most of the state of the art ways that robots use computer vision would generally see this as a bunch of pixels moving around. What this says to me is that perhaps we need a completely new approach to activity recognition. In a standard approach you first find a human, then track the human and identify their moving body parts, and then compare the way the parts are moving around to pre-existing models of various human activities. But clearly if I can so easily attribute human intentional action to squares and triangles, there is a much simpler feature space in which robots should be reasoning about intentional action.

I’m excited about the work of Dare Baldwin at the University of Oregon, she is working to uncover what low-level spatio-temporal features infants might be attending to when they correctly interpret intentional action.  Which at the very least can provide inspiration if not a detailed roadmap for building some intentional action recognition into our social robots.

December 10th, 2009 Posted by | HRI | no comments

Robots Podcast

The Robots podcast describes themselves as “the podcast for news and views on robotics. In addition to insights from high-profile professionals, Robots will take you for a ride through the world’s research labs, robotics companies and their latest innovations.”

Their most recent edition is on robot learning, where I was interviewed about social learning and my lab at Georgia Tech, and Sethu Vijayakumar is interviewed about his lab and the distinctions between human and machine learning.

They have more than thirty topical episodes of interviews with more than 60 roboticists over the past two years.  It’s quite a fun to browse through.

November 23rd, 2009 Posted by | Announcements, HRI, Machine Learning | no comments