Social Robotic Companion
The Huggable™ is a new type of robotic companion being developed at the MIT Media Lab for healthcare, education, and social communication applications. The Huggable™ designed to be much more than a fun interactive robotic companion. It is designed to function as a team member that is an essential member of a triadic interaction. Therefore, the Huggable™ is not designed to replace any particular person in a social network, but rather to enhance that human social network.
It is featured with a full body sensitive skin with over 1500 sensors, quiet back-drivable actuators, video cameras in the eyes, microphones in the ears, an inertial measurement unit, a speaker, and an embedded PC with 802.11g wireless networking. An important design goal of the Huggable™ is to make the technology invisible to the user. You should not think of the Huggable™ as a robot but rather as a richly interactive teddy bear. The actuators are designed to be silent and back drivable so as the Huggable™ moves, you do not hear or feel gears. The movements, gestures and expressions of the bear convey a personality-rich character, not a robotic artifact. A soft silicone-based skin covers the entire bear to give it a more lifelike feel and heft, so you do not feel the technology underneath. Holding the Huggable™ feels more like holding a puppy, rather than a pillow-like plush doll.
We worked with various Media Lab sponsors to create a series of Huggables for real-world applications and trials. We also collaborated with Microsoft Research, using Microsoft Robotic Studio to develop the communication avatar implementation.
We have developed a web interface to enable a remote operator (e.g., educator, grandparent, friend, etc.) to view the state of robot and evoke its behaviors. The web interface is a combination of a website and an application for streaming audio and video to and from the robot. The website includes a diagram that shows the robot’s body pose trajectory over time (via the potentiometer sensors), several buttons to execute different actions (movement and sound), an interface to enter text for the robot to speak via speech synthesis, and various check-boxes to toggle ON/OFF several of the aforementioned technologies. There are two video streams, one incoming from the robot’s video camera and another from the 3D virtual model of the robot. There is also a small animated cartoon used to indicate the whole-body gesture recognition state of the robot (e.g., whether it is being picked up, rocked, bounced, etc.). The operator can also talk to the user (i.e., the child interacting directly with the Huggable) through the robot’s speaker and listen to the user via microphones.
One challenge of teleoperation is providing the remote operator with adequate situational awareness of the context surrounding the robot. In particular, the robot’s camera often has a much narrower field of view than that of human peripheral vision. This gives the operator a sense of tunnel vision. It is also an issue in social interaction between user (child) and the Huggable because user assumes the Huggable has human-like field of view.To cope with this issue, we have implemented a “stale panorama” interface. To build the stale panorama, the robot autonomously captures video frames and associated head angle as it looks around the room. The captured frames are then projected to a much larger canvas. The result is a collage of still images that present the remote operator with a panorama of the environment.
The remote operator receives a real-time video stream from the robot’s eye camera as indicated by the “gaze window” on the GUI. The operator can direct the robot’s gaze to a new location by dragging the “target gaze” window to a different location in the stale panorama. The robot then performs IK to reposition the robot’s head so that the real-time video feed shown through the gaze window co-aligns with the target gaze window.
We have also developed a wearable interface that consists of a set of motion capture devices that the remote operator wears to stream his/her body posture and gestures to the Huggable.We have two methods: direct control or gesture recognition. The human operator wears a set of orientation sensor units for his/her head and the arms while holding a Wii Remote and a Nunchuk in both hands. Gestures performed via Wii Controllers can be recognized as abstract gestures such as ‘waiving arms’, ‘holding arms up’, ‘holding arms forward’ and sent to the robot to mimic the same gestures. Alternatively, the orientation information captured via arm-strapped sensors can be utilized to directly drive the robot’s arms and neck.
The sympathetic interface consists of a waldo-like device that maps to the Huggable robot’s body and joint angles. As the remote operator moves the waldo of the Huggable, joint angle positions are streamed in real-time to command the joint motors of the Huggable robot. For instance, the user can directly control the gaze and pointing direction of the robot by moving the waldo-Huggale arms and neck.
Robot Communication Avatar
In a social communication application the triad includes the Huggable™, a remote family member, and the child. For instance, the family member may be a parent who is away on a business trip, or a grandparent who lives far from the child. The Huggable™ enables a richer, multi-modal interaction — supporting communication and play through touch, shared space, vision and speech. The remote family member interacts with the child through the Huggable™ — controlling the semi-autonomous robot via a website and seeing and hearing the child through the eyes and ears of the Huggable™.
This video highlights the role of MSRS in our implementation of the robot communication avatar application.
Early Education Companion
In a distance education application, this triad includes the Huggable™, the student, and the teacher. Here the Huggable™ serves as a semi-autonomous robotic communication avatar that a remotely located teacher controls via the internet to interact with a student in an educational activity. The teacher can see the child through the Huggable’s™ cameras, hear the child through the microphones, talk to the child through the speaker, and gesture and express via animations the Huggable™ can perform. The Huggable™ can locally process how the child is touching it, picking it up, etc. and relay this information back to the educator.
In a healthcare application, the interaction triad includes the Huggable™, a member of the hospital or nursing home staff, and the patient or resident. Here the fully autonomous Huggable™ interacts with the patient to provide therapeutic benefit of a companion animal, and can also communicate behavioral data about this interaction to the nursing staff to assist them in promoting improved well-being of the patient.To serve as new type of robotic companion for therapeutic applications our design goals are:
- To be viscerally and emotionally pleasing to interact with, both with respect to how it feels to touch and how it responds to people.
- To provide measurable health benefit to people, especially health benefits that arise from touch and social support.
- To be a useful tool for the nursing staff or other care providers that augments existing animal assisted therapy programs (if present).
- To be a computationally flexible platform that allows us to explore other health related.
One important and novel capability we are developing for the Huggable™ is its ability to participate in active relational and affective touch-based interactions with a person. Social-relational touch interactions play a particularly important role for companion animals in their ability to provide health benefits to people. Touch can convey a wide variety of communicative intents — an animal can be tickled, petted, scratched, patted, rubbed, hugged, held in ones arms or lap just to name a few. To be effective, therapeudic robotic companions must also be able to understand and appropriately respond to how a person touches it — e.g., communicating with the right kind of emotive expression or performing and appropriate touch response such as nuzzling.
We have carried out initial experiments to assess the ability of the skin and somatic perceptual algorithms to classify the affective content of touch. A neural network was implemented to recognize nine classes of affective touch – tickling, poking, scratching, slapping, petting, patting, rubbing, squeeze, and contact. Each of these classes were again combined into six response types – teasing pleasant, teasing painful, touch pleasant, touch painful, punishment light, and punishment painful. The response type defines how the Huggable™ interprets the intent of the touch and what behavior to perform in response. For example, a pleasant touch should signify a happy reaction while strong punishment should result in a pain response.
“The Huggable”: Static Display of V2 Huggable Prototype. Star Wars Where Science Meets Imagination. International Touring Exhibit, 2008.
“The Huggable”: Interactive Demonstration of Third Generation Prototype at the San Raffaele Del Monte Tabor Foundation (HSR), Milan, Italy, May 6-7, 2008.
“The Huggable”: Interactive Demonstration of Second Generation Prototype at the Space Between: Making Connections during Palliative Care Conference Sponsored by the Highland Hospice, Inverness, Scotland, November 8th-9th, 2007.
“The Huggable”: Interactive Demonstration of Second Generation Prototype at the “Our Cyborg Future?” Exhibition as part of the Designs of the Time 2007 Festival, Newcastle, UK, October 19th, 2007
“The Huggable”: Interactive Demonstration of Second Generation Prototype at the AARP Life@50+ Conference, Boston, MA, September 6th-8th, 2007.
“The Huggable”: Interactive Demonstration of Second Generation Prototype at the Robots at Play Festival, Odense, Denmark, August 23rd-25th, 2007.
“The Huggable”: Static Display and Interactive Touch Sensor Panel as part of the “Our Cyborg Future?” Exhibition as part of the Designs of the Time 2007 Festival, Newcastle, UK, August 10th-October 27th, 2007.
“The Huggable”: Booth at the World Healthcare Innovation and Technology Congress Washington, DC, November 1st-3rd, 2006.
“The Huggable”: Interactive Technology Demonstration at Disney New Technology Forum: Best of SIGGRAPH 2006 at the Walt Disney Studios in Burbank, CA September 8th, 2006.
“The Huggable”: Interactive Technology Demonstration in Emerging Technologies Pavillion at SIGGRAPH 2006, Boston, MA, July 30th-August 3rd, 2006.
“The Huggable”: Static Display and Interactive Touch Sensor Panel as part of the “Tech’ing it to the Next Level: Highlights from iCampus, the MIT-Microsoft Alliance” Exhibition, MIT Museum, Cambridge, MA, May 23rd – December, 2006.
“The Huggable”: Booth and Focus Groups at The Digital Future – Creativity without Boundaries Conference, Aviemore, Scotland, May 11th, 2006
“The Huggable”: Technology Demonstration at the IEEE Consumer Communications and Networking Conference, Las Vegas, NV, Jan 9-10, 2006. W.D. Stiehl, J. Lieberman, C. Breazeal, L. Basel, R. Cooper, H. Knight, L. Lalla, A. Maymin, and S. Purchase.
“The Huggable”: Technology Demonstration at Microsoft Research Faculty Summit, Microsoft Conference Center, Redmond, WA, July 19th, 2005.
The Huggable™ is funded by the Things That Think (TTT) and Digital Life (DL) consortia, a Microsoft iCampus grant, a Highlands & Islands Enterprise grant, and other generous donations from MIT philanthropists.
Team Huggable™ Alumni
Cheng Hau Tong
Personal Robots Group
Professor Cynthia Breazeal
Jun Ki Lee
Heather-Marie C Knight
Kristopher Dos Santos
The Distance Lab (Highlands & Islands Enterprise)
Plush Bear Design
- Jeong, S., Logan, D., Goodwin, M., et al. (To appear). Challenges Conducting Child-Robot Interaction Research in a Pediatric Inpatient Care Context. The First Workshop on Evaluating Child-Robot Interaction held in conjunction with the Seventh International Conference on Social Robotics.
- Jeong, S., Dos Santos, K., et al. (2015) Designing a socially assistive robot for pediatric care. Proceedings of the 14th International Conference on Interaction Design and Children). ACM, New York, NY, USA, 387-390.
- Jeong, S., Logan, D., et al. (2015). A Social Robot to Mitigate Stress, Anxiety, and Pain in Hospital Pediatric Care. In Adams, J. A., & Smart, W. (Eds.), 2015. Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction Extended Abstracts. ACM, New York, NY, USA.
- Sooyeon Jeong, Developing a Social Robotic Companion for Pediatric Care for Stress and Anxiety Mitigation., 2014, M.Eng. Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology.
- Kristopher Dos Santos, The Huggable: A socially assistive robot for pediatric care., 2012. S. M. Media Arts and Sciences, MIT.
- Walter Dan Stiehl, Jun Ki Lee, Cynthia Breazeal, Marco Nalin, Angelica Morandi, Alberto Sanna (2009). “The Huggable: a Platform for Research in Robotic Companions for Pediatric Care”. IDC 2009: 317-320.
- J.K. Lee, R. Toscano, D. Stiehl, C. Breazeal (2008). “The Design of a Semi-Autonomous Robot Avatar for Family Communication and Education”. Proceedings of the 17th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN-08). Munich, Germany.
- Jun Ki Lee, Affordable Avatar Control System for Personal Robots, 2008. S.M. Media Arts and Sciences, MIT.
- Nicolina Akraboff, Design of Transmission Mechanisms for the Head of the ‘Huggable’ Robotic Teddy Bear., January 2008. Department of Mechanical Engineering, Massachusetts Institute of Technology.
- Robert Toscano, Building a Semi-Autonomous Sociable Robot Platform for Robust Interpersonal Telecommunication, May 2008, M.Eng. Department of Electrical Engineering and Computer Science.
- Heather Knight, An Architecture for Sensate Robots: Real Time Social-Gesture Recognition using a Full Body Array of Touch Sensors, September 2008, M.Eng. Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology.
- Levi Lalla, A Design of Actuation Mechanisms for Use in the “Huggable” Robotic Teddy Bear., June 2006. Department of Mechanical Engineering, Massachusetts Institute of Technology
- Stiehl, W.D., Breazeal, C., Han, K., Lieberman, J., Lalla, L., Maymin, A., Salinas, J., Fuentes, D., Toscano, R., Tong, C. H., and Kishore, A. (2006). The Huggable: A New Type of Therapeutic Robotic Companion. In ACM SIGGRAPH 2006 Sketches (Boston, Massachusetts, July 30 – August 03, 2006). SIGGRAPH ‘06. ACM Press, New York, NY, 14.
- Stiehl W.D., Lieberman, J., Breazeal, C., Basel, L., Cooper, R., Knight, H., Lalla, L., Maymin, A. & Purchase, S. (2006) “The Huggable: A Therapeutic Robotic Companion for Relational Affective Touch.” In Proceedings of IEEE Consumer Communications and Networking Conference (CCNC). Las Vegas, NV. Volume 2, 1290-1291.
- Stiehl, W.D. & Breazeal, C. (2005) “Affective Touch for Robotic Companions.” Proceedings of Affective Computing and Intelligent Interaction (ACII-05). Bejing, China.
- Stiehl, W., Lieberman, L., Breazeal, C., Basel, L., Lalla, L., & Wolf, M. (2005), “Design of a Therapeutic Robotic Companion for Relational, Affective Touch.” In Proceedings of Fourteenth IEEE Workshop on Robot and Human Interactive Communication (Ro-Man-05), Nashville, TN. 408-415. Best paper Award.
- Stiehl, W.D., Lieberman, J., Breazeal, C., Basel, L., Lalla, L. and Wolf, M. (2005). The Design of the Huggable: A Therapeutic Robotic Companion for Relational, Affective Touch. Proceedings of AAAI 2005 Fall Symposium on Caring Machines. Washington D.C.
- A Talking Teddy Bear Practicing in the Pediatric Hospital
- A Blue Robotic Bear to Make Sick Kids Feel Less Blue
- Teddy tech: Rise of MIT’s robot care bear Huggable
- Robot teddy to help sick children
- Stiehl, W. Chang, A., Wistort,R., Breazeal, C. “The Robotic Preschool of the Future: New Technologies for Learning and Play”, Finalist in the Como 4 Children Competition at Interaction Design for Children. (2009)
- Huggable: €10k Robots at Play Prize 2006.
- Stiehl, W. & Breazeal, C. (2005) “Design of a Therapeutic Robotic Companion for Relational, Affective Touch,” In Proceedings of Fourteenth IEEE Workshop on Robot and Human Interactive Communication (Ro-Man). Best Paper Award.