The new wave of driver assistance systems requires high performance image processing, but without sacrificing flexibility.

By Robert Green and Karen Parnell, Xilinx

Safety is the number one concern of vehicle customers. Safety equipment has evolved from the physical to the electronic domain, starting with advancements in tire and braking technology, through side impact protection and airbags, and on to today's 'driver assistance systems'. There is also a growing need to provide a wider range of information to the driver without taking attention away from actual driving, with voice activation and hands-free communication being key technology drivers. The latest vehicles are electronics-rich and sensor-based to continuously evaluate the surroundings, display relevant information to the driver and, in some instances, even take control of the vehicle. They hold great promise in increasing the safety, convenience and efficiency of driving.
Driver assistance systems can offer basic safety features, such as adding infrared (IR) cameras for improved visibility. More advanced equipment warns of potentially dangerous situations using a wider array of sensors, enabling the vehicle to be aware of surrounding traffic, lane direction and possible collision objects. The ultimate aim is for the vehicle to be able to react to this information automatically so that its occupants are kept safe, assisting the driver with information and car control. For example, video cameras have been installed in some of the latest trucks to capture images of the lane ahead. If the vehicle changes lanes without using indicators, maybe because the driver is suffering from tiredness, an alert is sounded through the cabin loudspeakers.

Driver assistance can also offer better levels of comfort by removing mundane driving actions. For example, conventional cruise control allows the driver to set a constant travel speed that can be overridden manually when needed. This has now evolved into Adaptive Cruise Control (ACC) which automatically controls the throttle and brakes to match the speed of the vehicle in front and keep a safe distance from it. If the vehicle ahead accelerates away or changes lane, ACC returns to the pre-set speed of the conventional cruise control.

Another emerging safety feature is 'passive occupant detection systems' driven by the US government mandate that states that all cars from model year 2006 onwards will be able to deploy airbags based on the passenger profile. Using cameras and other sensors for inputs, sophisticated occupant classification algorithms and extensive signal processing allow the vehicle airbag controller to variably deploy or suppress the passenger airbag based on height and proximity to the airbag.

Figure 1. Concept of FPGA in ACC Driver Assistance System

FPGAs in Driver Assistance Systems

Figure 1 shows a conceptual diagram of a Xilinx Field Programmable Gate Array (FPGA) in an ACC Driver Assistance System.

The system is partitioned into very high-speed input processing and relatively low speed sensor inputs and output control signals, each under the control of its own processor (e.g. a Xilinx MicroBlaze™ 32-bit soft processor or even an embedded IBM PowerPC™ in Virtex-II Pro™ FPGAs). The high-speed section is dedicated to the real-time processing of video coming from the cameras mounted at the front of the vehicle. Real-time processing is absolutely critical due to the nature of the application — crash avoidance, emergency procedures and alerts. Usually two or more cameras will be used to allow the capture of a stereo image, thus enabling calculation of image depth (directly related to real distances from objects) in the FPGA. When combined with radar and laser measurements, plus the information collected from gyros and wheel sensors to detect motion, a very accurate map of the vehicle's surroundings and its path through it can be calculated. Using fully flexible FPGAs, as opposed to off-the-shelf video components, equipment manufacturers can easily develop unique, optimized edge detection, image depth and enhancement algorithms that will differentiate the system performance from that of competitors. Capturing and processing this information in real-time requires the use of math intensive Digital Signal Processing (DSP) algorithms. However, software processing can't meet the performance requirements and although conventional DSP processors are an alternative, it often needs multiple devices to perform such high-speed tasks. Even ASSP video processors often cannot compare to the extremely high-speed DSP performance of Xilinx FPGAs, also known as XtremeDSP™ processing. This is the result of FPGAs having a fundamentally parallel processing structure, rather than being limited to the traditional serial processing found in traditional DSPs and software (comparison in Figure 2).

Figure 2. Xilinx FPGAs Offer Unrivalled DSP Performance

After processing the video, the decision tree mechanisms can be partitioned between hardware, for speed-critical algorithms like sudden object avoidance, and processor software, for sounding alerts such as lane drift warnings. Partitioning speed-critical processes into FPGA hardware also enables testing at real-time rates which is impossible in software. This algorithmic acceleration is also a key requirement of passive occupant detection systems, as described earlier.

As well as real-time performance, the reprogrammability of Xilinx FPGAs also offers superb system flexibility, enabling algorithm upgrades to be made even after deployment. This is important, as current driver support systems are still in the early days of research and development. As edge and object detection algorithms are improved over time, hardware upgrades can be done in a matter of minutes and no board redesign is required.

Figure 3

Bridging Networks with Programmable Interfaces

As the vehicle evolves into a truly networked area, the equipment manufacturer somehow needs to determine which standard will be the most successful or offer them the greatest advantage over other network protocols. Various network technologies have emerged that cover different requirements in the car, ranging from multimedia and personal communications networks in the cockpit such as Media Oriented Systems Transport (MOST) to car control networks like FlexRay. In Figure 2, a pre-verified Controller Area Network (CAN) interface core was chosen as an example, but this could be one of a wide range of standard or proprietary interfaces.

One such emerging networking protocol for use in cockpit communications is the Bluetooth™ wireless technology, a low-cost, low-power, short-range radio technology for mobile devices and for WAN/LAN access points. Bluetooth radio technology appears to be the ideal solution for in-car communications as the radio uses a fast acknowledgement and frequency-hopping scheme to make the link robust. Bluetooth radio modules avoid interference from other signals by hopping to a new frequency after transmitting or receiving a packet.

A driver will be able to use a Bluetooth cordless headset to communicate with a cellular phone in his or her pocket, for example. As a result, and in combination with voice activated dialing, driver distraction can be reduced and safety increased. While stationary (e.g. in the garage) Bluetooth wireless connections could also be used for connecting in-dash MP3 players to portable computing devices or a home network "base station" to enable the download of new songs. The automotive industry has created a special-interest group (SIG) for the definition of Bluetooth car profiles. The SIG includes such members as AMIC, BMW, DaimlerChrysler, Ford, GM, Toyota and VW.

Adding Bluetooth capabilities to in-built communications or multimedia systems is made much easier if they are based on a programmable platform FPGA. Integration of a Bluetooth radio module can be accelerated using a Xilinx FPGA as a simple bridging or baseband processing function between the RF components and the main system. To make this even easier, NewLogic Technologies, Inc. has recently introduced the BOOST Lite™ baseband processor and development board for Bluetooth technology. Based on the Xilinx Virtex FPGA, the BOOST Lite processor is designed as an "out of the box" Bluetooth wireless solution and can be integrated with any other third-party system.

IQ Solutions for Automotive Applications

To address the needs of automotive electronics equipment designers Xilinx has created a new range of devices with an extended industrial temperature range option. Called the "IQ" range, it comprises of current Xilinx industrial grade (I) FPGAs and CPLDs qualified to a new extended temperature grade (Q) as shown in Table 1.

Table 1. Temperatures supported by Xilinx Products
Temperature Grade/Range°
Product C I Q
FPGA TJ = – 40 to +85 TJ = – 40 to +100  TJ = – 40 to +125
CPLD TA = 0 to +70 TA = – 40 to +85 TJ = – 40 to +125

The first products qualified to operate at the new temperature grade are Spartan-XL™ 3.3V FPGAs ranging from 5K gates to 30K gates, and the 36 and 72 macrocell XC9500XL™ 3.3V CPLDs. Over the coming months the IQ range will be expanded to include FPGA devices up to 300K gates and CPLDs up to 512 macrocells in density, as shown in Table 2.

Table 2. Xilinx IQ Solutions Silicon for Automotive Applications
Xilinx IQ Solutions Silicon Selector
Product Family Packages Voltage Density Range
XC9500XL CPLDs VQ44, VQ64, TQ100 3.3 V 36 - 72 Macrocells
CoolRunner XPLA CPLD VQ44, VQ100, TQ144, PQ208 3.3 V 32 - 512 Macrocells
CoolRunner II CPLD VQ44, VQ100, TQ144, PQ208 1.8 V 32 - 512 Macrocells
Spartan XL FPGA VQ100, TQ144, PQ208, BG256 3.3 V 5K - 40K Gates
Spartan II FPGA TQ144, PQ208, FG256 2.5 V 15K - 200K Gates
Spartan IIE FPGA TQ144, PQ208, FT256, FG456 1.8 V 50K - 300 K Gates


The new wave of driver assistance systems requires high performance image processing, but without sacrificing the flexibility required during early stages of research and development of object detection and automotive network technologies. The use of Xilinx FPGAs at the heart of such systems offers the industry's best DSP performance and unrivalled support for network connectivity standards and gives system architects a fully flexible design platform to work on. By enabling these systems to work in real-time, it is now possible to provide emergency driver alerts or assisted car control and significantly increase safety. In addition, by enabling wireless hands-free communications, using low cost Xilinx devices as bridges between RF components and the main system, FPGAs are a key technology for ensuring driver safety.

For further information please visit