Jenga—that game that needs careful use of agile and deft fingers, as well as careful consideration of which block can be removed without toppling the tower. Yup, that game! A robot can play that game.
Engineers at the Massachusetts Institute of Technology developed a self-learning robot with a soft-pronged gripper, a force-sensing wrist cuff, and an external camera, which it uses to see and feel the structure of the Jenga tower and its individual blocks. The details of the Jenga-playing smart machine were published in the journal Science Robotics.
The machine-learning approach of the Jenga-playing robot could help robots in assembling small parts in a manufacturing line like mobile phones
The Jenga-playing Robot
With the information it receives from its camera, the robot can analyze and compare this information to the moves it has previously made. It considers the outcomes of those moves to make a calculation about which individual piece could be moved and how much force is needed to be used to remove the piece successfully. The robot “learns” in real-time about the best moves for different situations in the game.
Alberto Rodriguez, assistant professor in the Department of Mechanical Engineering at MIT, said the Jenga-playing smart machine achieved something that cannot be done by previous smart systems:
“Unlike in more purely cognitive tasks or games such as chess or Go, playing the game of Jenga also requires mastery of physical skills such as probing, pushing, pulling, placing, and aligning pieces. It requires interactive perception and manipulation, where you have to go and touch the tower to learn how and when to move blocks. This is very difficult to simulate, so the robot has to learn in the real world, by interacting with the real Jenga tower. The key challenge is to learn from a relatively small number of experiments by exploiting common sense about objects and physics.”
How the Jenga-playing Robot learned to Play
For a robot to learn how to play Jenga, it would need at least tens of thousands of block-extraction attempts as initial data. The self-learning robot, however only needed about 300. The researchers used a more data-efficient way to let the robot to learn how to play Jenga. It is inspired by human cognition, the way we ourselves learn and approach the game.
The team used a customized industry-standard ABB IRB 120 robotic arm. It was first trained with 300 random extraction attempts on the Jenga tower. With each attempt, a computer would record the associated visual and force measurements of the move and mark each of them with a success or failure.
It clusters one attempt with many similar measurements and outcomes to represent one behavior to increase efficiency. This is the same way in which humans cluster similar behavior.
“The robot builds clusters and then learns models for each of these clusters, instead of learning a model that captures absolutely everything that could happen,” said Nima Fazeli, MIT graduate student and lead author of the paper.
Beyond Playing Jenga
This robot is not just a playing companion for those who cannot find someone to play Jenga with. The researchers highlighted that its ability to quickly learn the best way to carry out a task, not just from visual cues but from real situations, could be applied to creating robots that can do tasks that need careful physical interaction.
“In a cellphone assembly line, in almost every single step, the feeling of a snap-fit, or a threaded screw, is coming from force and touch rather than vision,” Rodriguez said. “Learning models for those actions is prime real-estate for this kind of technology.”
