Engineering news
The Johns Hopkins University robot – known as Spot – learned to do things such as stacking blocks, with the usual training timeframe of a month reduced to just days.
By using positive reinforcement, an approach familiar to anyone who has used treats to change a dog's behaviour, the team of computer scientists dramatically improved the robot's skills and did it quickly enough to make training robots for real-world work more feasible. The findings are published in a new paper called “Good Robot!”
Lead author Andrew Hundt said: “The question here was how do we get the robot to learn a skill? I've had dogs, so I know rewards work, and that was the inspiration for how I designed the learning algorithm.”
Unlike humans and animals that are born with highly intuitive brains, computers are blank slates and must learn everything from scratch. True learning is often accomplished with trial and error, however, and roboticists are working out how robots can efficiently learn from their mistakes.
The team accomplished that by devising a reward system that works for a robot in the same way that treats work for a dog. While a dog might get a biscuit for a job well done, the robot earned numeric points.
To stack blocks, Spot needed to learn how to focus on constructive actions. As it explored the blocks, it quickly learned that correct behaviours for stacking earned high points, but incorrect ones earned nothing. Reaching out but not picking up a block, for example, received no points, as did knocking over the stack. Spot earned the most by placing the last block on top of a four-block stack.
The team was able to reduce the practice time by first training a simulated robot, then running tests with Spot.
“The robot ‘wants’ the higher score,” Hundt said. “It quickly learns the right behaviour to get the best reward. In fact, it used to take a month of practice for the robot to achieve 100% accuracy. We were able to do it in two days.”
Positive reinforcement also helped teach the robot to play a simulated navigation game. The ability to learn from mistakes in all types of situations is critical for designing a robot that could adapt to new environments, the researchers said.
“At the start the robot has no idea what it's doing, but it will get better and better with each practice. It never gives up and keeps trying to stack, and is able to finish the task 100% of the time,” Hundt said.
The findings could help train household robots to do cleaning or laundry, useful tasks for assisting elderly people at home. The researchers said it could also help to improve self-driving cars.
“Our goal is to eventually develop robots that can do complex tasks in the real world – like product assembly, caring for the elderly and surgery,” Hundt said. “We don't currently know how to program tasks like that – the world is too complex. But work like this shows us that there is promise to the idea that robots can learn how to accomplish such real-world tasks in a safe and efficient way.”
The research was published in IEEE Robotics and Automation Letters.
Want the best engineering stories delivered straight to your inbox? The Professional Engineering newsletter gives you vital updates on the most cutting-edge engineering and exciting new job opportunities. To sign up, click here.
Content published by Professional Engineering does not necessarily represent the views of the Institution of Mechanical Engineers.