TECH

Artificial brains also sleep

Machines that need to sleep
Computer scientists, especially those involved with artificial intelligence, like to ask themselves if, in the future, androids will dream of robotic sheep.
No one knows the answer, but those artificial intelligences of the future will almost certainly need periods of rest, which will offer them benefits similar to those that sleep provides to living brains.
It turns out that neural networks become unstable after continuous periods of self-learning. What has now been discovered is that they can return to stability after being exposed to states that simulate sleep, suggesting that even artificial brains need to doze off occasionally.
Yijing Watkins and his colleagues at the Los Alamos National Laboratory, in the USA, exposed neural networks to states analogous to the waves that living brains experience during sleep. This stabilized the networks after they started to "get lost" in continuous periods of unsupervised learning.
"It was as if we were giving neural networks the equivalent of a good night's sleep," said the researcher.

Pulsed neural networks
The curious discovery came when the team worked to develop neural networks that approximate how humans and other biological systems learn to see. The group initially struggled with the stabilization of simulated neural networks, subjected to unsupervised dictionary training, which involves classifying objects without having previous examples to compare them.
"The question of how to prevent learning systems from becoming unstable really arises only when trying to use biologically realistic neuromorphic processors, or when trying to understand biology itself," said Professor Garrett Kenyon. "The vast majority of machine learning, deep learning and artificial intelligence researchers never encounter this problem because, in the very artificial systems they study, they have the luxury of performing global mathematical operations that have the effect of regulating the overall dynamic gain of system".
The team's next goal is to implement its sleeping algorithm on Intel's Loihi neuromorphic chip. They hope that allowing Loihi to sleep from time to time will allow it to process information stably from an artificial retina in real time.
If the results confirm the need for sleep by artificial brains, we can probably expect the same for androids and other intelligent machines that may arise in the future.

Authors: Yijing Watkins, Edward Kim, Andrew Sornborger, Garrett T. Kenyon