【Introduction】Traditionally, computing tasks in automobiles are performed by microcontroller units (MCUs) and application processors (APs). A typical mid-range car can contain 25 to 35 MCUs/APs, while luxury cars may use 70 or more. More and more cars require extremely complex and computationally intensive functions for tasks such as advanced driver assistance systems (ADAS), infotainment, control, networking and safety. Many of these applications involve machine vision in the form of image and video processing, as well as artificial intelligence (AI).
Traditionally, computing tasks in an automobile were performed by a microcontroller unit (MCU) and an application processor (AP). A typical mid-range car can contain 25 to 35 MCUs/APs, while luxury cars may use 70 or more. More and more cars require extremely complex and computationally intensive functions for tasks such as advanced driver assistance systems (ADAS), infotainment, control, networking and safety. Many of these applications involve machine vision in the form of image and video processing, as well as artificial intelligence (AI).
It is difficult for a single processor architecture to handle all the electrical interfaces and protocols required by peripherals such as sensors, cameras and displays. Furthermore, in many cases, these processors simply cannot meet the extreme computational demands of tasks such as machine vision and artificial intelligence.
To address this complexity, designers of automotive systems are turning to Field Programmable Gate Arrays (FPGAs), not to replace existing MCUs/APs, but to act as a bridge between them and other devices, and by offloading communication and other computationally intensive tasks.
Because FPGAs can be programmed to support a variety of electrical interfaces and protocols, they can act as a bridge between the MCU/AP and sensors, cameras, and displays. Additionally, because they can perform computations and operations in massively parallel fashion, FPGAs can be used to perform computationally intensive vision processing and AI tasks, freeing up host processors for other activities.
This article discusses the processing requirements of modern vehicles and describes some of the automotive applications that FPGAs can address. It then introduces some example FPGAs from Lattice semiconductor and shows how they can be used to address connectivity, processing and security issues. An associated development board is also provided to help designers get started.
Target Automotive Applications for FPGAs
To support their ADAS capabilities, today’s cars employ many sensors outside the vehicle, including cameras, radar, lidar, and ultrasonic detectors. In many cases it is necessary to acquire data from different sensors, preprocess this data (remove noise and format it as needed), and use sensor fusion to combine the data so that the resulting information has low uncertainty Because if data from different sensors will be used separately. In many cases, AI applications are used to analyze data, make decisions, and take appropriate action.
A relatively new trend is the deployment of Electronic (also known as “digital”) rearview mirrors. In this case, a wide-angle high-resolution camera is installed in the rear window. The video stream from this camera is shown on a digital Display, replacing conventional mirrors, resulting in a clear rear view unobstructed by rear-seat passengers. In some cases, the video stream from the rear-view mirror-mounted camera may be merged with the video stream from the rear-window camera. The three pieces of information are “stitched together” to provide a single image presented on the ultra-wide electronic mirror, giving the driver a higher degree of situational awareness of what is happening around the vehicle.
Another recent trend is the deployment of in-vehicle cameras on the dashboard, on the steering column or integrated into the rearview mirror (regular or electronic). When combined with AI, these interior mirrors can be used for a variety of tasks, such as recognizing who is sitting in the driver’s seat and adjusting the mirrors and mirrors accordingly. In addition to monitoring drivers to ensure they are paying attention to the road and not dozing off, such a system could look for signs of drowsiness, as well as medical problems or distress, such as seizures or heart attacks, and take appropriate action. These actions may include activating hazard warning lights, applying the brakes, and steering the vehicle to the curb.
For video-based applications, in some cases, a single video input needs to be split into multiple streams; in other cases, the design requirement may be to aggregate multiple video streams into one.
With the growing popularity of electric vehicles (EVs), there is a need to monitor and control the electric motor, as well as monitor and manage the charging process for maximum battery life.
On top of that, many of today’s cars are starting to support 5G or V2X, where V2X (“Vehicle to Anything”) refers to the communication between the vehicle and any other entity that may affect (or be affected by) the vehicle, from the road side infrastructure to other vehicles. In addition to this connectivity, security is also required to prevent vehicles from being hacked.
Automotive Grade Equipment
It is important to remember that not all FPGAs are suitable for automotive applications. The Automotive Electronics Council (AEC) was originally an organization formed by Chrysler, Ford, and GM in the 1990s to establish common parts qualification and quality system standards. One of the most frequently cited AEC documents is AEC-Q100, “Qualification for Stress Test of Integrated Circuits Based on Failure Mechanisms.”
IATF 16949:2016 is a technical specification designed to develop a quality management system that provides continuous improvement, emphasizes defect prevention, and reduces variation and waste in the automotive industry’s supply chain and assembly processes. IATF 16949:2016 is based on the ISO 9001 standard and was created by the International Automotive Task Force (IATF) and ISO technical committees.
Electronic system suppliers in the automotive market are increasingly requiring semiconductor suppliers to provide AEC-Q100 compliant products and be able to demonstrate ISO/TS-16949 certification of their quality systems.
Choosing the Right FPGA for the Job
FPGAs are very flexible, but different device families offer various combinations of capabilities and capabilities that make them more suitable for specific tasks. For example, in embedded vision applications, modern cameras and displays typically use the MIPI interface. Both MIPI CSI-2 (camera/sensor) and DSI (display) protocols use a communication physical layer (PHY) called D-PHY. Legacy MCUs/APs may not support this interface, but some FPGAs, such as Lattice Semiconductor’s CrossLink-NX Embedded Vision and Processing FPGAs.
Automotive system developers looking to evaluate CrossLink-NX FPGAs will find the combination of the LIFCL-VIP-SI-EVN CrossLink-NX VIP Sensor Input Board (Figure 1) and the LF-EVDK1-EVN Modular Embedded Vision Kit interesting (the former can as an input pad for the latter). In addition to the CrossLink-NX FPGA, the sensor input board features four 13-megapixel Sony IMX258 CMOS MIPI image sensors supporting 4K2K @ 30 frames per second (fps) or 1080p @ 60 fps. It also supports easy sensor connection via three independent PMOD interfaces.
Figure 1: The CrossLink-NX VIP Sensor Input Board, available as an input to the Embedded Vision Development Kit, contains a
CrossLink-NX FPGA and supports aggregation of four MIPI Sony IMX258 image sensors. (Image credit: Lattice Semiconductor)
For compute-intensive applications that also require high I/O bandwidth, such as AI for tasks such as gesture recognition and control, speech recognition and control, human presence detection, occupant recognition, and driver monitoring, Lattice’s ECP5 FPGAs have up to 3.2 Gbit/Serializer/Deserializer (SERDES), up to four channels per device in dual-channel block for higher granularity, up to 85K Lookup Tables (LUTs), Enhanced Digital Signal Processing (DSP) block , which provides a 2x resource improvement for symmetric filters, as well as Single Event Disturbance (SEU) mitigation support. These FPGAs also provide programmable I/O support for LVCMOS 33/25/18/15/12, XGMII, LVTTL, LVDS, Bus-LVDS, 7:1 LVDS, LVPECL, and MIPI D-PHY I/O interfaces.
An example ECP5 device is the LFE5U-85F-6BG554C, which has 84,000 logic elements, 3.75 megabits (Mbits) of RAM, and 259 I/Os. The LFE5UM-45F-VERSA-EVNG ECP5 Versa Development Kit is also of interest (Figure 2). Available in a half-length PCI Express (PCIe) form factor, the board allows designers to evaluate key connectivity features of ECP5 FPGAs, including PCIe, Gigabit Ethernet (GbE), DDR3, and general-purpose SERDES performance.
Figure 2: The ECPe Versa Development Kit is available in a half-length PCI Express form factor, allowing designers to evaluate key connectivity features of ECP5 FPGAs, including PCIe, GbE, DDR3, and general-purpose SERDES performance. (Image credit: Lattice Semiconductor)
Securing a car with an FPGA
Security threats from hackers are increasing, and new breaches are constantly occurring. In the case of automobiles, cyberattacks can lead to loss of control, resulting in injury or death to passengers and pedestrians, as well as damage to cars, other vehicles and property.
A big part of an automotive security solution is establishing a Root of Trust (RoT); that is, a hardware resource in a system that can always be trusted. One solution is an FPGA-based hardware RoT (HRoT), such as that provided by Lattice’s MachXO3D family of devices. In addition to large LUT resources and high I/O, these flash-based devices offer instant-on and hot-plug capabilities. Common applications include glue logic, bus bridging, bus interface, motor control, power-up control, and other control logic applications.
Of particular interest, the MachXO3D is the only FPGA with both dual-boot capability and less than 10K LUTs, equipped with a National Institute of Standards and Technology (NIST) certified hard immutable security engine. This allows the MachXO3D to act as an HRoT for the car in the form of a system power-on-then-off device. Once the system is powered up, the MachXO3D first checks to make sure it is running verified firmware. Then it checks the firmware of other devices in the system. If any component in the system (including itself) is attacked or compromised, MachXO3D rejects the suspect firmware and reloads that component with a known-good, verified firmware image.
For developers interested in evaluating MachXO3D-based designs, the LCMXO3D-9400HC-D-EVN MachXO3D development board provides an extensible prototyping platform (Figure 3). The board features an L-ASC10 (Analog Sensing and Control) hardware management device, general purpose I/O for Arduino and Raspberry Pi boards, two Hirose FX12-40 header locations (DNI), Aardvark headers (DNI ) and 128 Mbit Serial Peripheral Interface (SPI) flash with quad read capability.
Figure 3: The MachXO3D development board features a MachXO3D FPGA, an L-ASC10 (analog sensing and control) hardware management device, support for Arduino and Raspberry Pi boards, two Hirose FX12-40 header locations (DNI), an Aardvark header and USB-B connection for device programming. (Image credit: Lattice Semiconductor)
The board features a 4 x 6 inch form factor with a USB mini-B connector for power and programming, as well as multiple header locations to support Arduino, Aardvark, FX12, Hirose, and Raspberry Pi. Comes with USB cable and quick start guide.
Modern automotive electronics require more and more sensors, electrical interfaces and protocols, with corresponding demands on processing power and bandwidth. Artificial intelligence and machine vision processing and increased security requirements complicate solution implementation using classic MCU or AP approaches.
As shown, with the proper application of FPGAs, designers can add a degree of flexibility and processing power to bridge disparate processing environments, perform sensor aggregation and fusion functions, meet I/O bandwidth requirements, and in a massively parallel fashion Perform calculations and operations while freeing the host processor for other activities.
On the security front, a flash-based FPGA with dual-boot capability and a NIST-certified immutable security engine can act as the car’s HRoT and ensure it and other devices run only authenticated firmware, preventing hackers from cryptographically breaching the car’s systems .