Teledyne FLIR Releases Free Expanded Starter Thermal Dataset for advanced automotive systems and Autonomous Vehicle Testing
Development work in the automotive industry on advanced safety systems and greater levels of vehicle autonomy has a heavy reliance on sensors of all kinds including cameras, LiDAR, radar and thermal imaging. Using these kinds of sensors, the vehicle control system has the ability to detect people, objects and animals in the roadway and can take appropriate action if required.
Rarely can a single type of sensor provide sufficient information on its own to enable the split second decision making required and so autonomous vehicles rely on sensor fusion, the combination of outputs from multiple sensors to create a more complete picture of what lies in the road ahead or around the vehicle.
Thermal cameras are an important input to the fused sensor data as they are able to provide information in conditions that can provide a lack of or absence of data from other sensors, for example in the dark or in foggy conditions. The challenge however, remains as to how the vehicle control system interprets thermal imaging data to recognise different categories of objects and whether something is human or animal.
Expanded Dataset
To overcome this challenge, IR and thermal imaging specialist, Teledyne FLIR released a dataset that categorised images. Now, the company has released an expanded free thermal dataset for Advanced Driver Assist Systems (ADAS) and self-driving vehicle researchers and developers.
The expanded starter dataset nearly doubles the original, industry-first free dataset and features more than 26,000 annotated images from the US, England, and France in day and night time conditions. It also triples the number of categories and now includes person, bike, car, motorcycle, bus, train, truck, traffic light, fire hydrant, street sign, dog, skateboard, push-chair, scooter and other vehicle labels. The expanded free starter thermal imaging dataset allows the automotive and academic community to quickly evaluate the vehicle safety algorithm performance, neural network testing, and thermal sensors, such as the FLIR ADK.
According to Chris Posch, director of automotive engineering at Teledyne FLIR, the originally released free thermal dataset has been used by developers and as a tool in academic research to explore safety of automated driving technology with thermal imaging whilst the expanded dataset, with more images and categories, will further enhance detection and classifications models, especially focused on automatic braking capabilities as part of a multispectral, sensor-fused system.
“When combined with visible light cameras, LIDAR and radar, thermal sensors help create a comprehensive and redundant system to identify and classify roadway objects using sensor fusion data. Testing has demonstrated that thermal sensors are uniquely capable of seeing pedestrians, large animals and other vulnerable road users in conditions where current automatic emergency braking (AEB) systems are challenged, including in total darkness, most fog, smoke, shadows, inclement weather, and sun or headlight glare,” says Posch.
In an ongoing effort to make roads safer, twenty automakers committed to the US Congress that all newly manufactured cars and trucks will be equipped with AEB systems by September 1st, 2022. Furthermore, high profile testing and demonstrations from safety organisations including the AAA in America, demonstrate the need for more effective and intelligent systems to improve vehicle safety, especially in challenging conditions where thermal cameras excel.
With the potential for millions of thermal-enabled AEB systems on vehicles, Teledyne FLIR anticipates the cost of its thermal sensor to continue to decrease significantly, which will encourage wide-scale adoption and ultimately enable safer vehicles and roads.
- UK manufacturing steps up to COVID-19 crisis - April 2, 2020
- Clustering Innovation - March 12, 2020
- A Global Monitor - March 6, 2020