Exploiting Lightweight Statistical Learning for Event-Based Vision Processing
ABSTRACT
This paper presents a lightweight statistical learning framework potentially suitable for low-cost event-based vision systems, where visual information is captured by a dynamic vision sensor (DVS) and represented as an asynchronous stream of pixel addresses (events) indicating a relative intensity change on those locations. A simple random ferns classifier based on randomly selected patch-based binary features is employed to categorize pixel event flows. Our experimental results demonstrate that compared to existing event-based processing algorithms, such as spiking convolutional neural networks (SCNNs) and the state-of-the-art bag-of-events (BoE)-based statistical algorithms, our framework excels in high processing speed (2_ faster than the BoE statistical methods and >100) faster than previous SCNNs in training speed) with extremely simple online learning process, and achieves state-of-the-art classification accuracy on four popular address-event representation data sets: MNIST-DVS, Poker-DVS, Posture-DVS, and CIFAR10-DVS. Hardware estimation shows that our algorithm will be preferable for low-cost embedded system implementations.
EXISTING SYSTEM :
Computer vision systems are pervasively applied in modern society including security surveillance , human-machine interface , assisted driving , and industrial automation. Traditional computer vision systems are equipped with frame-based image sensors. These systems scan and process all the pixels in every frame snapshot by the sensor at a fixed frame rate (e.g. 30 frames per second), regardless of the light intensity change between successive frames. These systems may waste considerable computational power and time to process redundant visual information. On the other hand, these frame-based systems are unable to respond on-the- fly to fast-changing scenes as they have to wait for the next frame tic. In general, the traditional vision systems are not suitable for those applications requiring low system cost, low power consumption and immediate response to environmental dynamics. To address this problem, the frame-less dynamic vision sensor (DVS) is proposed in recent literature .
DVS is inspired by the visual path structures and functionalities found in the biological visual systems. In typical biological visual systems, neurons asynchronously code and communicate the visual information as spatiotemporally sparse light change in the form of spikes. To mimic such highly-efficient biological mechanism, each pixel in a DVS sensor continuously and independently monitors the change of light intensity cast onto it. When the relative intensity change of one pixel reaches a threshold, the pixel immediately sends out a spike affiliated with its coordinate (address) in the pixel array, as well as the change polarity (increase/decrease in intensity). One pixel sends out information only when perceivable intensity changes are detected due to visual motion or light condition variations. Otherwise it remains inactive to avoid generating redundant data for static scenes. Moreover, each pixel asynchronously responds to its local intensity change on-the-fly (usually several microseconds in latency) without waiting for the other pixels’ outputs to synchronize an image frame. Therefore, the output of DVS is a time-continuous stream of spikes encoded with pixel addresses and intensity change polarity, called address-event representation (AER). Compared to frame-based image sensors, DVS features higher power efficiency and low response latency, and is believed to be very suitable for embedded systems.
PROPOSED SYSTEM :
This work proposes a lightweight statistical learning framework based on random ferns for AER data classification. Compared with existing methods for AER classification,our framework exhibits higher processing speed, less computational resource requirement, simplest online learning process and state-of-the-art classification accuracy.In general, the traditional vision systems are not suitable for those applications requiring low system cost, low power consumption and immediate response to environmental dynamics. To address this problem, the frame-less dynamic vision sensor (DVS) is proposed in recent literature . DVS is inspired by the visual path structures and functionalities found in the biological visual systems. In typical biological visual systems, neurons asynchronously code and communicate the visual information as spatiotemporally sparse light change in the form of spikes. To mimic such highly-efficient biological mechanism, each pixel in a DVS sensor continuously and independently monitors the change of light intensity cast onto it. When the relative intensity change of one pixel reaches a threshold, the pixel immediately sends out a spike affiliated with its coordinate (address) in the pixel array, as well as the change polarity (increase/decrease in intensity). One pixel sends out information only when perceivable intensity changes are detected due to visual motion or light condition variations. Otherwise it remains inactive to avoid generating redundant data for static scenes. Moreover, each pixel asynchronously responds to its local intensity change on-the-fly (usually several microseconds in latency) without waiting for the other pixels’ outputs to synchronize an image frame.
CONCLUSION :
This work proposes a lightweight statistical learning framework based on random ferns for AER data classification. Compared with existing methods for AER classification , our framework exhibits higher processing speed, less computational resource requirement, simplest online learning process and state-of-the-art classification accuracy. The software simulation results and the estimated hardware performance demonstrate that our statistical framework is a promising candidate for high-speed low-cost embedded AER systems. Our future work will focus on: 1) exploring more lightweight statistical learning methods with further improved performance for AER data processing, and 2) implementing those methods on customized system-on-chips with integrated AER sensors.