Action analysis and control strategy for rat robot automatic navigation

A rat robot is an animal robot, where a rat is connected to a machine system via a brain-computer interface. Electrical stimuli can be generated by the machine system and delivered to the rat's brain to control its behavior. The sensory capacity and flexible motion ability of rat robots highlight their potential advantages over mechanical robots. There are two challenges of rat robot automatic navigation. The first challenge is to recognize the action status of the rat robot, which is an essential feedback for determining the stimuli/instructions for it to accomplish certain movements. The second challenge is the design of the automatic instruction model that steers the rat robot to perform navigation. Due to inherent characteristics and instincts of the rats, the controlling strategy of the rat robots is different from mechanical robots. In this thesis, we propose a new idea for analyzing the action states of the rat robot. A miniature camera is mounted on the back of the rat robot and the egocentric video captured by the camera is used to analyze its action. We propose two action analysis methods. The first method is based on an optical flow algorithm and the second method incorporates deep neural networks. We propose two automatic instruction models. The first model learns from manual control data to mimic the human controlling process, and the second model issues instructions according to human experts' knowledge. We build a rat robot and apply these models to enable it to navigate in different scenes automatically. In order to produce more accurate optical flow estimation, we propose a row convolutional long short-term memory (RC-LSTM) network to model the spatial dependencies among image pixels. Our RC-LSTM network is integrated with Convolutional Neural Networks and achieves competitive accuracy. To analyze potentially more complex actions from the egocentric videos, we extend our deep neural networks used for rat states analysis to be a two-stream architecture. A spatial attention network is incorporated to help our model to focus on the relevant spatial regions to recognize actions. Our model is evaluated on two egocentric action recognition datasets and achieves competitive performance.