Sense, Think, and Act on Level 3 Autonomous Drive Capabilities Today

There has been a buzz around autonomous vehicles in the automotive sector for years now, but a dearth of enabling development tools has limited their deployment in any meaningful way. As a result, many automakers and Tier 1 suppliers have been relegated to theoretical research and development, while others still have been stalled at lower levels of driving automation. However, this year we released the Automated Drive Kit (ADK), a software enablement platform for the development and testing of autonomous drive solutions. Currently, the ADK integrates multiple sensors, automotive-grade processors, the Robot Operating System (ROS), and functional safety software tools in a commercially available, off-the-shelf platform. Automotive stakeholders now have a baseline for building Level 3 autonomous drive capabilities today, and a roadmap to full autonomy for the future.

SAE International standard J3016 defines five levels of autonomous driving. They are:

Level 1: Driver Assistance – The vehicle provides a specific function, such as speed control or steering
Level 2: Partial Automation – The vehicle assists in functions like steering, acceleration, and braking
Level 3: Conditional Automation – The vehicle can drive itself, but the driver has to monitor and be able to take over at any time
Level 4: High Automation – The vehicle can take over driving on surface streets, but not highways or freeways
Level 5: Full Automation – The vehicle takes over driving completely, allowing drivers to remain in a “mind-off” state

Vehicles with Level 2 capabilities are already on the road today. However, there are major challenges moving to Level 3, mainly because of the complexities associated with passing control between vehicles and drivers. A lack of development tools and platforms further complicates matters for autonomous drive engineers. As a result, many automakers and ride-share companies are working independently on the end-to-end autonomous drive hardware and software stack in an attempt to be first to market with commercially viable Level 3 capabilities. Progress has been slow to say the least.

For most automotive engineers, the goal of their autonomous drive development efforts is to create algorithms that define how self-driving vehicles respond in safety-critical situations. The value is not, however, in integrating the underlying sensors, processors, and firmware that gather input data and execute these algorithms.

To accelerate the development and deployment of Level 3 autonomous drive capabilities and beyond, automotive stakeholders need platforms that abstract the complexity of enabling technologies and allow them to focus on the end application software. In response, NXP Semiconductors, in conjunction with AutonomouStuff, introduced the Automated Drive Kit (ADK) at CES 2018.

Accelerating Level 3 Autonomous Vehicles with the Automated Drive Kit

The ADK is suite of integrated hardware and software solutions that speed time to market of Level 3 autonomous vehicle designs. The development platform includes a number of modular components that can be swapped in or out depending on design goals (Figure 1) including:

Figure 1. NXP Semiconductors’ Automated Drive Kit (ADK) is a comprehensive set of technology building blocks that accelerates the development and deployment of Level 3 autonomous vehicles. Included in the ADK is the NXP BlueBox 2.0 – BLBX2-DB; LiDAR, radar, vision, and positioning sensors; and the Robot Operating System (ROS) and related middleware.

Self-driving vehicle capabilities can be broken into three functional segments: Sense, Think, and Act (Figure 2). Of these, automotive engineers at OEMs and Tier 1 suppliers are mostly concerned with developing algorithms applied during the “Act” phase, but depend on reliable Sense and Think infrastructure in order to do so. This is what the ADK provides.

Figure 2. The core of the ADK is the NXP BlueBox 2.0 platform, which has a performance-to-power ratio of 90,000 DMIPS at less than 40 watts. Pictured here is a diagram of how BlueBox performs Sense, Think, and Act functions associated with self-driving vehicles.

Sense – Data streams from the ADK’s myriad radars and cameras enable applications such as pedestrian detection, object detection, traffic sign recognition, lane departure warning (LDW), and smart headlamp beam control. These radar, LiDAR, vision, and V2X inputs are supported by the NXP BlueBox 2.0’s onboard vision and radar processing elements (S32V234 and SR32R27, respectively). Image data, for example, can be processed in one of two forms on the S32V234 processor: encoded (MJPEG or H.264) via Ethernet or in raw format via the MIPI CSI-2/VIU interfaces.

Think – The NXP BlueBox 2.0’s onboard processors provide 90,000 DMIPS of performance in a 40 W power envelope for executing autonomous drive workloads, which is supported by 16 GB of built-in DDR4 RAM and a 256 GB SSD. During the Think stage, sensor data is transferred to the LS2084A processor for analysis of the driving environment.

Act – After analyzing the driving environment, autonomous drive algorithms running on the LS2084A processor are used to determine the best, safest course of action, such as accelerating, steering, or braking.

To support automotive engineers in their algorithm development efforts, the ADK is compatible with Baidu’s Apollo Software Platform, as well as the aforementioned LOP modular and Neusoft ADAS vision software packages. Autonomous drive designers can also choose to integrate their own proprietary algorithms with the ADK platform.

Regardless, AUTOSAR-compliant, functional safety-certified software tools such as the S32 Design Studio integrated development environment (IDE) outfitted with a vision software development kit (SDK) and an MCU abstraction layer within the OS serve to abstract intricacies of requisite Level 3 autonomous drive hardware.

Level 3 Today, Full Autonomy Tomorrow

NXP Semiconductor’s ADK is available off-the-shelf from AutonomouStuff, providing easy access to an open innovation platform for autonomous drive systems that integrates all of the necessary components for Sense, Think, and Act functionality.

The NXP ADK is accelerating time-to-market for Level 3 self-driving cars today, but a technology roadmap will deliver new features that support Level 4 and 5 capabilities in the near future. Whereas autonomous vehicle development had been stuck in neutral, platforms like the ADK now pose this question to automakers and Tier 1 suppliers: Will you be left behind?

Mark O'Donnell
Mark O'Donnell
Mark O'Donnell has nearly 20 years in semiconductor marketing. Projects range from the first 32-bit MCU with embedded flash for automotive to the first ISO 26262-certified MCU. He leads the SafeAssure program for functional safety and speaks at industry events on a range of automotive design topics. Passionate about the future of mobility and the significant opportunities ahead, he routinely answers simple but challenging questions from his children: “Dad, I think cars that drive around by themselves is okay—but when will they fly?”

Comments are closed.

Buy now