TY - JOUR
T1 - Exploring the Viability of Bypassing the Image Signal Processor for CNN-Based Object Detection in Autonomous Vehicles
AU - Cahill, Jordan
AU - Parsi, Ashkan
AU - Mullins, Darragh
AU - Horgan, Jonathan
AU - Ward, Enda
AU - Eising, Ciaran
AU - Denny, Patrick
AU - Deegan, Brian
AU - Glavin, Martin
AU - Jones, Edward
N1 - Publisher Copyright:
© 2013 IEEE.
PY - 2023
Y1 - 2023
N2 - In the field of autonomous driving, cameras are crucial sensors for providing information about a vehicle's environment. Image quality refers to a camera system's ability to capture, process, and display signals to form an image. Historically, 'good quality' in this context refers to images that have been processed by an Image Signal Processor (ISP) designed with the goal of providing the optimal experience for human consumption. However, image quality perceived by humans may not always result in optimal conditions for computer vision. In the context of human consumption, image quality is well documented and understood. Image quality for computer vision applications, such as those in the autonomous vehicle industry, requires more research. Fully autonomous vehicles inevitably encounter constraints concerning data storage, transmission speed, and energy consumption. This is a result of enormous amounts of data being generated by the vehicle from suites made up of multiple different sensors. We propose a potential optimization along the computer vision pipeline, by completely bypassing the ISP block for a class of applications. We demonstrate that doing so has a negligible impact on the performance of Convolutional Neural Network (CNN) object detectors. The results also highlight the benefits of using raw pre-ISP data, in the context of computation and energy savings achieved by removing the ISP.
AB - In the field of autonomous driving, cameras are crucial sensors for providing information about a vehicle's environment. Image quality refers to a camera system's ability to capture, process, and display signals to form an image. Historically, 'good quality' in this context refers to images that have been processed by an Image Signal Processor (ISP) designed with the goal of providing the optimal experience for human consumption. However, image quality perceived by humans may not always result in optimal conditions for computer vision. In the context of human consumption, image quality is well documented and understood. Image quality for computer vision applications, such as those in the autonomous vehicle industry, requires more research. Fully autonomous vehicles inevitably encounter constraints concerning data storage, transmission speed, and energy consumption. This is a result of enormous amounts of data being generated by the vehicle from suites made up of multiple different sensors. We propose a potential optimization along the computer vision pipeline, by completely bypassing the ISP block for a class of applications. We demonstrate that doing so has a negligible impact on the performance of Convolutional Neural Network (CNN) object detectors. The results also highlight the benefits of using raw pre-ISP data, in the context of computation and energy savings achieved by removing the ISP.
KW - Bayer filter
KW - Object detection
KW - autonomous vehicles
KW - image signal processor
KW - neural networks
KW - raw data
UR - http://www.scopus.com/inward/record.url?scp=85159645661&partnerID=8YFLogxK
U2 - 10.1109/ACCESS.2023.3270710
DO - 10.1109/ACCESS.2023.3270710
M3 - Article
AN - SCOPUS:85159645661
SN - 2169-3536
VL - 11
SP - 42302
EP - 42313
JO - IEEE Access
JF - IEEE Access
ER -