In this article, you’ll learn:

• • 6 levels of autonomous cars
  • History of autonomous vehicles
  • How do self-driving cars work?
  • Security vulnerabilities, risks & concerns of connected and autonomous vehicles
  • What company makes the security technology for ADAS?
  • Moving the flood of data in autonomous vehicles
  • What company makes automotive MIPI solutions?
  • Additional resources

6 levels of autonomous cars

In 2014, the engineering group SAE International created six levels of driving automation which have since been adopted by the U.S. Department of Transportation. These include:

• Level 0: No automation
• Level 1: Driver assistance
• Level 2: Partial driving automation
• Level 3: Conditional driving automation
• Level 4: High driving automation
• Level 5: Full driving automation

Automakers have already announced Level 3 autonomous driving cars—and are working to develop and deploy Level 4 self-driving trucks as well as commercial robotaxis. According to Accenture, vehicles with full-on self-driving capabilities could start hitting highways as early as 2030.

Keep on reading: SAE level of automation in cars simply explained.

History of autonomous vehicles

The concept of a self-driving car was first introduced by General Motors (GM) at a 1939 World Fair exhibit. In 1953, RCA Labs and GM built a miniature car that was guided and controlled by patterned wires. A full-size system was subsequently demonstrated in Nebraska using specially designed road lights and buried detector circuits.

By the 1980s, self-driving technology had improved considerably, with the Defense Advanced Research Projects Agency (DARPA) leveraging lidar, computer vision, and autonomous robotic controls to direct a vehicle at speeds of up to 19 miles per hour (31 km/h). DARPA partner HRL Laboratories later demonstrated the first off-road map and sensor-based autonomous navigation in a test vehicle that traveled over 2,000 feet (610 m) at 1.9 miles per hour (3.1 km/h) through challenging terrain.

The 1990s saw Carnegie Mellon University pioneer and refine neural networks to steer and control autonomous vehicles under the auspices of the Navlab project, with a test vehicle completing a 3,100 miles (5,000 km) cross-country journey. Additional self-driving advances were made by major automakers and technology companies like Waymo, Uber, and Tesla throughout the 2000s, 2010s, and early 2020s. In 2014, Tesla Motors announced its first version of Autopilot, which later expanded to support autonomous steering, braking, speed adjustment, and parking capabilities. In October 2020, Tesla rolled out the first version of its full self-driving beta (FSD Beta) software and continues to release updates at a steady cadence.

How do self-driving cars work?

Most advanced driver-assistance systems (ADAS) powering vehicles with various levels of autonomy leverage a combination of specialized cameras and sensors to create an internal map of the vehicle’s surroundings. These sensors include:

• Lidar—Pulses thousands of beams of infrared laser light at objects to calculate distances and avoid objects.
• Radar—Uses radio waves to measure angles, ranges, and velocities of objects in most environmental conditions.
• Sonar—Identifies large objects made of solid materials, such as metal and ceramics, at short distances.
• Inertial navigation system—Helps self-driving cars stabilize themselves.
• GPS—Geolocates with numerical coordinates, including latitude and longitude, while navigating by combining real-time GPS coordinates with other digital map data applications.

Self-driving cars analyze the data generated by these sensors to plot navigational paths and react in real-time by stopping, speeding up, slowing down, and avoiding objects. They reduce the risk of accidents and collisions by implementing safeguards, alerting drivers, and taking full control of a vehicle if necessary. Moreover, self-driving cars automatically detect and react to other vehicles, bicyclists, pedestrians, construction zones, potholes, traffic accidents, and traffic jams. Perhaps most importantly, self-driving cars enforce safety standards that may be deliberately or accidentally ignored by human drivers.

Security vulnerabilities, risks & concerns of connected and autonomous vehicles

Semiconductors in connected vehicles and self-driving cars power extremely complex electronic systems. In the past, vehicle electronic systems implemented flat architectures with isolated functions controlling various components of the powertrain and vehicle dynamics. These electronic systems communicated primarily through legacy bus interconnect protocols, such as controller area network (CAN) and media-oriented systems transport (MOST) technologies.

To support the realization of Level 4 and Level 5 (L4/L5) autonomous driving, a massive architectural shift is underway. The software-defined vehicle, automotive Ethernet, vehicle-to-everything (V2X) connectivity, over-the-air updates (OTAs), and domain controller units are just some of the technologies required to achieve L4/L5 capabilities.

Indeed, new electronic systems support powertrain and vehicle dynamics, ADAS, autonomous driving, connectivity, and infotainment. At the heart of these electronic systems is a complex, multi-island IC containing multi-core processing, dedicated artificial intelligence and machine learning (AI/ML) engines, mixed-signal processing, and more.

Whether it’s a complex system on chip or a mixed-signal IC sitting at a sensor edge, security and safety are essential. Indeed, the advancements in vehicle electronic systems have resulted in a large attack surface for adversaries to exploit. In commercial or industrial applications, security is focused on providing trust, protecting assets, and protecting identities. In automotive, these focus areas remain, but another dimension is added.

This is because security vulnerabilities have the potential to directly impact safety measures implemented in a vehicle. For example, the lack of a robust safety architecture can cause a design to malfunction, with failures creating new security penetration points in the ICs, systems, and throughout the entire vehicle. On the flip side, an incomplete security architecture may be exploited by adversaries to circumvent or disable safety features, making the vehicle vulnerable to run-time failures.

What company makes the security technology for ADAS?

Rambus automotive hardware security modules (HSMs) are designed to protect self-driving cars and connected vehicles. These HSMs provide secure boot, secure firmware (OTA) upgrades, secure debug, and work with other security functions such as MACsec, IPsec, and TLS embedded protocol engines that protect network traffic in cars. To operate properly, ECUs must run the manufacturer intended firmware—without tampering. A root of trust ensures firmware is valid and can be securely updated when needed. Rambus offers embedded HSM (root of trust) variants for both ASIL-B (RT-640) and ASIL-D (RT-645) that are specifically designed for the functional safety requirements of ISO 26262, an international automotive electronics system standard.

The Rambus RT-640 Embedded HSM recently received Automotive Safety Integrity Level B (ASIL-B) ISO 26262 certification. Certified ASIL-B compliance is a critical requirement for automotive manufacturers and their suppliers to ensure vehicle systems meet necessary safety levels.

Integrated into an automotive SoC, the ASIL B certified RT-640 silicon IP design provides powerful cryptographic functions, state-of-the-art safety mechanisms, and anti-tamper technologies to protect critical automotive electronics and data.

Moving the flood of data in autonomous vehicles

All the data from the array of cameras and sensors employed in autonomous vehicles takes tremendous bandwidth. The MIPI Camera Serial Interface 2 (MIPI CSI-2®) v3.0 is increasingly the workhorse solution for transporting this vast volume of data. CSI-2 v3.0 offers capabilities including Unified Serial Link (USL) for encapsulating connections between a sensor module and application processor.

Regarding the choice of network connectivity in electronic systems, weight is not normally a first-order consideration, but it absolutely is when it comes to vehicles. A major networking hurdle introduced by the proliferation of sensors is the weight of cabling. In many vehicles, wiring is one of the top four heaviest subsystems. This issue is compounded as more cars go electric adding in the weight of the battery. A Tesla® battery pack, for instance, weighs about 900 pounds which nets out much heavier than an engine and full tank of gas.

What’s more, it’s often the electric vehicle makers that are leading the charge for autonomous driving. They need more sensors and better networking while simultaneously reducing weight to compensate for the battery. The weight benefit of MIPI, and networking technology such as automotive Ethernet, is that it can provide low-latency, high-bandwidth connections with fewer wires than legacy networking solutions. This enables the continued profusion of sensors for ADAS while keeping the weight of cabling low.

On the human interface side, whether for instrumentation, navigation or entertainment, modern vehicles include a large and growing number of displays. Here too MIPI plays a leading role with DSI-2^® solutions providing the high-bandwidth, low latency connectivity needed for high-resolution video and images.

What company makes automotive MIPI solutions?

Rambus has been a provider of MIPI IP solutions since 2010 and offers 32 and 64-bit digital controllers for CSI-2 and MIPI DSI-2 connectivity. Partnering with top-tier MIPI C/D-PHY suppliers, such as Mixel and Samsung, Rambus solutions have enabled over 250 ASIC and FPGA MIPI designs. An increasing number of these designs are for ADAS applications with leaders in the automotive market which Rambus supports with a full suite of customization and integration services, applicable safety manual, FMEDA and DFMEA.

The Rambus MIPI CSI-2 Transmitter (TX) Controller and the associated development processes have been certified to meet the ISO 26262 functional safety requirements. As with the embedded HSM solutions discussed earlier, using ISO 26262 certified IP enables automotive chip makers to accelerate the process of creating and safety certifying their SoCs.

Additional Resources:

– Other blogs around automotive & security:

1. Autonomous Vehicles: Memory Requirements & Deep Neural Net Limitations
2. Automotive Security: Protecting vehicle electronic systems
3. NextChip Win Signals Growing Momentum for Rambus Automotive Security IP
4. Designing automotive memory
5. Addressing automotive security challenges with a hardware root of trust
6. The challenge of securing connected vehicles
7. How not to get pwned @ automotive cyber-security
8. No quick fix for automotive insecurity
9. Securing connected vehicles with Rambus CryptoManager
10. Autonomous vehicles shift security risks into overdrive

– White Paper: Navigating the Intersection of Safety and Security

– Market page: Automotive Solutions

– Products for Automotive Applications:

• RT-640 Embedded HSM ISO 26262 ASIL-B
• RT-645 Embedded HSM ISO 26262 ASIL-D
• MACsec Protocol Engines
• IPsec Protocol Engines
• MIPI CSI-2 Controller
• MIPI DSI-2 Controller

Gullo kicked off his Q&A session by observing that automotive security best practices currently fall into three primary categories: authentication, multi-faceted designs, and flexibility.

“Authentication needs to happen in both directions. In other words, the car has to trust the cloud and the cloud has to trust the car,” he told panel participants and conference attendees. “Unfortunately, I think that authenticating vehicles sometimes gets less attention than it should. This is also true for any IoT device, even refrigerators and washing machines.”

As Gullo pointed out, a multi-faceted design approach is required to address a range of threat vectors, including attacks on the cloud-to-car connection, the in-vehicle network and specific ECUs. However, he emphasized there isn’t a “single, simple solution” that offers optimal security.

“For example, the components for V2X security may not be effective for monitoring and protecting in-vehicle networks. In general, security architectures need to be flexible because future threats are unlikely to resemble our current understanding of threat vectors,” Gullo explained. “These architectures need to have the ability to learn, evolve, and improve ‘in the field’ as new threats emerge. We also need to be thoughtful regarding solution complexity so systems can be adapted quickly as new threats emerge. This means relying on the fundamentals, such as proven algorithms, robust key management, secure boot loaders and constant threat detection, for example.”

As Gullo noted, this is precisely why automotive security architecture needs to evolve from static, simple solutions to a more dynamic framework that is self-learning, easily updatable and multi-faceted to address multiple threat vectors. This progression inevitably brings a number of new issues to the fore, including end-to-end secure data storage for autonomous vehicles.

“There are a host of companies whose core competence is secure, cloud-based data storage. OEMs can and should leverage these companies, although they should make it clear that while partners are tasked with securely storing data, they don’t own it,” Gullo opined. “Analyzing the data, generating insights from the information and acting on those insights is solely within the purview of the OEMs. Also, it goes without saying that a robust key management solution is required to secure the data in the vehicle and during transmission to and from the cloud service.”

To be sure, there are expected to be more than 350 million connected cars on the road by 2020. Google’s autonomous vehicles generate about 1 gigabyte of data every second, while Intel says autonomous vehicle are likely to produce about 2 petabytes of data per year. Information generated by connected and autonomous vehicles includes environmental data, as well as vehicle and driver performance.

“Maintaining the integrity of safety-critical and forensic vehicle data, particularly with respect to V2X, driver performance and vehicle performance, is absolutely critical. While some data should be shared for the ‘common good,’ it will undoubtedly be challenging to reach consensus on precise parameters,” Gullo emphasized. “Whether it’s through the Auto-ISAC or some other consortium, the industry clearly needs to agree on a ‘common good’ data set and ensure that vehicle owners are aware of the requirement to share this information.”

Gullo also described current security standards, specifications and guidelines including the ISO 26262 standard for functional safety and SAE’s J3061 Cybersecurity Guidebook (for Cyber-Physical Vehicle Systems).

“There is also SAE’s pending J3101 standard titled Requirements for Hardware-Protected Security for Ground Vehicle Applications, while UMTRI and the Southwest Research Institute are working on a framework for secure OTA software and firmware upgrades. This space is still evolving, although quite a lot has already been accomplished,” he added.

Jill Ingrassia, the Managing Director of Government Relations and Traffic Safety Advocacy at AAA, told conference attendees the auto industry has managed to significantly reduce the amount of vehicle crashes in recent years. However, more progress must be made, as thousands of people a year still lose their lives in traffic accidents.

According to Ingrassia, connected vehicle technology can help reduce accidents. As the AAA exec noted, multiple intelligent transportations systems – already deployed in the field – have continued to evolve over the years. The next stage of ITS is expected to include advanced systems, such as lane departure and forward collision warnings, braking and parking assistance systems, as well as adaptive headlights. All will be designed to help counteract human error and tendencies.

Perhaps not surprisingly, Ingrassia acknowledged that the industry faces a myriad of challenges in designing and deploying fully autonomous vehicles. Indeed, automakers have entered a transitional stage between semi-autonomous and fully autonomous capabilities. This evolution, says Ingrassia, presents its own set of concerns.

According to Joe Gullo, the senior director for Rambus Ecosystem strategy and development, security is one primary concern the industry must immediately address for intelligent transportation systems. To be sure, modern vehicles are essentially a network of networks – packed with a range of embedded communication methods and capabilities.

“Of course there is broad consensus that vehicle cyber security ranks as a top priority for the automotive industry,” Gullo told Rambus Press during an interview on the sidelines of CES 2016. “Unfortunately, there are still no clearly defined vehicle security specifications. This is not a problem that will be going away soon. In fact, it will only get worse as more and more connected vehicle systems are manufactured and installed in the next generation of semi-autonomous cars and trucks.”

Potential vulnerabilities include altering over the air (OTA) firmware updates, unsecure vehicle-to-vehicle communication, the unauthorized collection of driver or passenger information, seizing control of critical systems such as brakes or accelerators, intercepting vehicle data and tampering with third-party dongles.

As Gullo emphasizes, adopting a hardware-first approach to security and implementing the necessary functionality on the SoC level is a key element of protecting intelligent transportation systems – both now and in a fully autonomous future.

“To avoid potentially dangerous scenarios, vehicles should be equipped with robust DPA countermeasures to protect against side-channel attacks,” Gullo added. “In addition, the automotive industry needs to shield vehicle peripherals and components against tampering, as well as provide secure OTA updates for various systems.”

[youtube https://www.youtube.com/watch?v=NfeCRxrtIJY]

Participants unveiled a number of LSS-based prototypes and explored additional use cases for the evolving technology.

frog

The frog team presented multiple LSS-powered prototype platforms that focused on three major categories, including eye tracking, vehicle sensing and intelligent roadways.

“Most eye tracking technology uses very advanced computer software that relies on high-definition cameras that are able to get a clear view of users eyes only when they focus on a specific location,” Carlos Elena-Lenz told attendees.

[youtube https://www.youtube.com/watch?v=lds0uVwyKDE]

“Today’s solutions are too large to be placed on glasses or on a person without drawing attention or being cumbersome. By virtue of its package and power advantages, LSS is an ideal solution.”

Elena-Lenz and his team also identified a number of specific applications for LSS-powered eye tracking devices, such as observing fatigue, enhanced night vision, assistive technology via gaze recognition and augmented/VR reality.

“Eye tracking covers a range of applications across industries,” he added. “Historically, the technology has been used for market research, although the advent of mobile devices and wearables has ushered in a breadth of new opportunities.”

A similar LSS-centric approach can also be applied to autonomous vehicles.

“As vehicles of all stripes realize different levels of autonomy – from self parking to self driving – the need for sensing technologies becomes more important especially in close proximity to people and physical objects,” he explained. “Meanwhile, the UAV market is growing rapidly as well. The UAV market alone is poised to drive market growth. Within that market, UAV sensing is expected to grow to be a $1B market by 2020.”

According to frog, the affordances of LSS span a wide range of applications, including toys, drones, military and commercial vehicles.

“Alternative technologies exist today, but lack important context,” said Elena-Lenz. “Ultrasonic sensors detect proximity, but not precise location. PIR also detects proximity, but can’t sense lateral movement.”

Specific LSS-based solutions for autonomous vehicles highlighted by frog included collision avoidance, navigation and aerial imaging.

“Collision avoidance is applicable across many vehicle sizes from toys, drones, cars and trucks,” he explained.

[youtube https://www.youtube.com/watch?v=V1eob7_0xss]

“Smaller vehicles gain the ability to detect and navigate – while larger vehicles utilizing LIDAR for long-range mapping can still use LSS for close quarters navigation around people or physical objects.”

Beyond avoiding collisions as a result of detecting proximity and presence, LSS’s ability to sense movement enables autonomous navigational systems to rapidly react and choose a viable direction.

“Consider a drone equipped with multiple LSS which allows it to select a vector in three dimensions – thereby avoiding multiple objects moving towards it with varying velocities,” Elena-Lenz told summit participants. “In terms of aerial imaging, primary use cases include mining, construction, infrastructure management, agriculture and emergency response scenarios.”

The frog team also believes LSS-powered smart streetlamps can play a potential role in the evolution of intelligent roadways by tracking traffic flow, monitoring congestion, vehicle speeds and alerting authorities to motorists in need of roadside assistance.

[youtube https://www.youtube.com/watch?v=OOqqCdVp4p8]

“Connecting a cities roads at scale is a massive undertaking, especially when doing it retroactively. LSS’s low cost, small footprint, and low power consumption make it an ideal candidate,” Elena-Lenz opined. “LSS provides optical flow, image change detection and near IR all in a small package; capabilities ideally suited for the large scale sensing of transportation infrastructure.”

In addition to presenting LSS-powered prototype platforms, the frog team outlined a number of potential use cases for Rambus’ lensless smart sensor technology including real-time, thermal performance monitoring of commercial solar modules; industrial automation (part presence); embedded battery monitoring and intelligent agriculture.

IXDS

According to Prof. Reto Wettach, Rambus lensless smart sensors have the potential to fulfill three primary roles:

* Upgrade – To upgrade existing products.
* Integral – To be part of an original product design.
* Auxiliary – To observe (monitor) other objects for change.

Wettach also identified six primary categories Rambus LSS technology could potentially help evolve, such as smart infrastructure (cities and homes), tool manufacturing, medical, toys, consumer electronics and professional equipment. Specific LSS-powered prototypes showcased at the event included an assembly & maintenance platform, a self-driving model vehicle and a smartwatch.

[youtube https://www.youtube.com/watch?v=RLXLK1r_ZOs]

Eliott Jones, Rambus Vice President of User Experience, who led the effort from the Rambus side, feels particularly strongly about the upgrade potential that Wettach referenced. He explained, “Obviously, most of the world of objects already exists. So, if LSS can be incorporated into them as a retrofit due to its low power and minute size, there is huge potential to bring intelligence to existing domains such as city infrastructure and vehicle navigation, among others.”

The IXDS team outlined three key features of the assembly & maintenance platform, including low energy sipping, extended battery life and a minimal price point (disposable).

As Wettach notes, a number of platforms and devices could potentially benefit from the low energy sipping requirements of LSS, such as smart cat-eyes, structural integrity monitors for urban infrastructure (bridges or buildings) and traffic lights. Meanwhile, the low cost of LSS would allow the technology to be deployed in disaster relief emergencies (non-retrievable items), disposable medical devices and pay-per-usage monitoring and packaging platforms.

Similarly, the IXDS self-driving model vehicle on display at the summit emphasized three primary LSS features: size, cost and symbol recognition capabilities.

[youtube https://www.youtube.com/watch?v=KQ9GQFgrMxo]

According to Wettach, Rambus’ uber-mini lensless smart sensors can also be integrated in internal medical tools, implants and wearables. In addition, the symbol recognition capabilities of LSS could be used to track license plates, read barcodes and assist drivers with navigation.

Gesture control and low energy sipping were the primary themes of the LSS-based prototype smartwatch designed by IXDS – which could ultimately be powered by kinetic energy at some point in the future. In the meantime, the gesture control capabilities of LSS offer potential benefits for in-car infotainment systems, smart devices and music players.

[youtube https://www.youtube.com/watch?v=E4juMytG6aU]

Interested in learning more about Rambus lensless smart sensors? You can check out our official LSS page here and article archive here.