People rarely use just one sense to understand the world, but robots usually only rely on vision and, increasingly, touch. Carnegie Mellon University researchers find that robot perception could improve markedly by adding another sense: hearing.
In what they say is the first large-scale study of the interactions between sound and robotic action, researchers at CMU’s Robotics Institute found that sounds could help a robot differentiate between objects. Hearing also could help robots determine what type of action caused a sound and help them use sounds to predict the physical properties of new objects.
“A lot of preliminary work in other fields indicated that sound could be useful, but it wasn’t clear how useful it would be in robotics,” said Lerrel Pinto, who recently earned his Ph.D. in robotics at CMU and will join the faculty of New York University this fall. He and his colleagues found the performance rate was quite high, with robots that used sound successfully classifying objects 76 percent of the time. The results were so encouraging, he added, that it might prove useful to equip future robots with instrumented canes, enabling them to tap on objects they want to identify.
To perform their study, the researchers created a large dataset, simultaneously recording video and audio of 60 common objects—such as toy blocks, hand tools, shoes, apples and tennis balls—as they slid or rolled around a tray and crashed into its sides. They have since released this dataset, cataloging 15,000 interactions, for use by other researchers.
Though the size of this dataset is unprecedented, other researchers have also studied how intelligent agents can glean information from sound. For instance, Oliver Kroemer, assistant professor of robotics, led research into using sound to estimate the amount of materials, such as rice or pasta, by shaking a container, or estimating the flow of those materials from a scoop.
Pinto said the usefulness of sound for robots was therefore not surprising, though he and the others were surprised at just how useful it proved to be. They found, for instance, that a robot could use what it learned about the sound of one set of objects to make predictions about the physical properties of previously unseen objects. “I think what was really exciting was that when it failed, it would fail on things you expect it to fail on,” he said. For instance, a robot couldn’t use sound to tell the difference between a red block or a green block. “But if it was a different object, such as a block versus a cup, it could figure that out.”
1. About the findings of this study, which of the following is true?
A) Robots can predict actions that can cause a sound.
B) Robots cannot use hearing to understand the world.
C) Robots’ hearing is much sharper than that of humans.
D) Robots may use sound to distinguish different objects.
2. Compared with previous studies, the mentioned research ______.
A) developed more advanced robots
B) collected the largest amount of data
C) clarified the way to classify varied sounds
D) identified the objects robots can recognize
3. Oliver Kroemer is cited because he ______.
A) shared a large quantity of data
B) found a new research direction
C) carried out similar experiments
D) developed some intelligent agents
4. What does Pinto mean by saying “it would fail on things you expect it to fail on”?
A) Robots can succeed within the expected range.
B) The hearing ability of the robots is still very limited.
C) Robots were unable to differentiate colors of objects.
D) The researchers hoped for the failure of the experiment.
5. Which of the following would be the best title for the text?
A) To Equip Robots with “Ears”
B) A Breakthrough in Robotics
C) The Sensory System of Robots
D) Distinguishing Objects with Robots