Edge to cloud: Implementing human-centric data in automotive

Advances in biometric monitoring via the advancement of 5G networks mean there is a great opportunity for automotive industries to intersect with healthcare and biometric data industries. As an example, we show Red Hat using its unique building blocks to build the solutions for these opportunities alongside NTT DATA, a global IT services organization based in Japan.

We are currently in the middle of a fourth industrial revolution, one driven by information. This has led to an explosive increase in Internet of Things (IoT) technology.

For years, the performance of these devices was heavily dependent on their network connection, but with 5G, an optimized and scalable technology is now available to carry out extensive data acquisition and real-time analyses, and use the results for automated decision-making.

This is one of the factors driving the development of autonomous vehicles, which rely heavily on IoT and the strategic use of data. A 2021 report by the Society of Motor Manufacturers and Traders (SMMT) said, “Already more than half of new cars sold are available with at least one semi-autonomous driving feature and the vast majority have some form of connected technology.”²

However, because they lack the speed and stability of 5G connections, current automobiles generally do not have real-time access from the edge of the network to centrally located cloud compute power. This limits the possibilities for applications, but that’s changing with the move to 5G. However, using 5G to make that connection and open those possibilities takes time and expertise.

As current algorithmic car models focus on how safely we drive and how to predict drivers’ mistakes, they are also becoming health monitoring systems, creating a crossroads between the automotive and the healthcare industries. With the move to 5G networks, this data can be processed in new ways and applied in novel applications, and can lead to a truly human-centric design model for applications.

Because of this, vehicle manufacturers are sitting on a golden opportunity to be the first to cross this barrier, and are well positioned to build alliances with healthcare organizations and medical practitioners. 

Automotive ecosystems—including parts suppliers, dealerships, end customers, and service centers—benefit from deeper collaboration, better communication, and disintermediation, made possible by digital tools and technologies. These changes are driven in part by a convergence of information technology (IT) and operational technology (OT) initiated by companies within the automotive ecosystem.

As of mid-2022, all motor vehicles (including trucks, buses, vans, and sport utility vehicles) put on the European Union market will have to be equipped with these safety features:³

• Intelligent speed assistance.
• Alcohol interlock installation facilitation.
• Driver drowsiness and attention warning systems.
• Advanced driver distraction warning systems.
• Event data recorders.

To fulfill all of these, and similar proposed regulations in the United States⁴, vehicles will need to process tremendous amounts of biometric data.

At the forefront of this transformation, Red Hat is working collaboratively with leading partners such as NTT DATA to add in-vehicle intelligence, safety features, automated capabilities, and telematics functionality to enhance customer experiences in modern-day motoring.

One such project is the Human Driving Perception Platform (HDPP), which works to detect if the driver is feeling secure, safe, or under threat. The HDPP is an implementation focusing on a human-centered approach to design. Insufficient human-centered design can cause a misunderstanding of how humans naturally interact and operate vehicles.

Insufficient human-centered design (HCD) caused the misunderstanding of how humans naturally interact with and operate vehicles.

For automotive safety integrity level (ASIL) automation at levels 4 and 5 (levels that are indicative of self-driving with little human intervention), the traditional human-vehicle interaction needs to evolve, and new principles of human-centered design have to be reviewed, focusing on humans as intelligent, sensitive, and emotional beings relying on trust, safety, comfort, and efficiency. As such, automated vehicles must not just be able to react to conditions on the road to drive safely, but also must respond to how the humans inside the vehicle perceive that driving, and the overall emotional comfort of those passengers.

The importance of human driving perception, as multi-modal analysis from multiple sensor streams during every moment of driving is significant for continual understanding of driver’s mental and physical state, driving behavior and perception NTT DATA and Red Hat have contributed to HCD on the development of the Human Driving Perception Platform.

To understand how the technology behind this project is put together, first we must travel to the edge of the network.

Vehicles at the network edge 

The successful implementation of an IoT architecture, such as an automated vehicle system, must combine or integrate the edge computing approach with that of cloud computing in a suitable manner.

• Cloud computing is the act of running workloads within clouds—which are IT environments that abstract, pool, and share scalable resources across a network. 
• Edge computing describes the approach to data processing at the location where the data is generated—that is, decentralized, at the edge of the network, for example on the sensors or gateways themselves. Only relevant data or aggregated interim results are then sent from the edge and used for further central processing. A huge amount of data is generated at the edge of the networks, and there is a need to transfer this from multiple production sites to local datacenters and various cloud environments in order to use artificial intelligence (AI) or machine learning (ML) technologies in a centralized manner and gain data-driven insights.

Challenges and demand

In the case of automated vehicles, connecting the edge of the network to cloud computing was a challenge in the past, as network connections to vehicles were usually not robust enough to make the data transfer feasible.

However, the advent of 5G wireless technology has made the use of cloud computing connected to the edge significantly more viable for autonomous vehicles. This is true despite the fact that an IoT, cloud, and edge computing architecture can seem overwhelming at first glance. It seems overwhelming because it often contains a wide variety of device types, different partial architectures and protocols, various physical locations, different workload characteristics, multi- and hybrid-cloud environments, machine learning, data science, security across the entire stack, monitoring, and various other different technologies.

The complete solution, therefore, does not consist of one technology, product, or even architecture concept to guarantee successful implementation. The aim is to find a modular, scalable architecture that supports the need-oriented addition or removal of functions and already supports as many requirements and use cases as possible.

In the case of the HDPP, the challenge was how to get data being collected at the edge—namely real-time camera views of a driver’s face—and then get that data processed in the cloud in a timely manner so the systems at the edge are able to make use of it.

How the tech works: The HDPP validated pattern

The first component of the HDPP is a simulation with a validated pattern, a distributed cloud-native application that implements key aspects of a modern IoT architecture. This demo is based on Red Hat® OpenShift® Container Platform, a Kubernetes platform, and uses various middleware components optimized for a cloud-native usage. 

From a technical point of view, the validated pattern offers two core concepts: Bobbycars and Bobbycar zones. 

• Bobbycars: Bobbycars are vehicle simulators implemented in Quarkus (cloud-native Java™ stack) that simulate vehicles (connected cars) and send telemetry data to a regional IoT cloud backend. In this demo they represent the vehicle edge.
• Bobbycar zone: A Bobbycar zone represents a location-based configuration, such as an environmental zone, for which a maximum CO2 emission has been defined, or a listing of various mobility services that are made available at this location. The Bobbycar zones are implemented as Kubernetes custom resources. 

For each simulated vehicle, a route is selected at random from a pool of routes. Driving the route from start to end is simulated and the current position and current telemetry data such as speed, RPM, CO2, and emissions are sent to a regional IoT cloud backend infrastructure. This is done by streaming all sensor data from the vehicles via MQTT, an open messaging protocol, to local Kafka clusters.

Apache Kafka is the central system for all incoming data. The incoming data is made available to a real-time dashboard for visualization via websocket and updates a distributed in-memory cache. So the current status of the entire IoT system can be retrieved from the cache at any time. 

How to use Apache Camel-K to integrate platform components

In order to integrate all components for HDPP, the cloud-native integration framework of Apache Camel-K is used. Specifically, the integration of MQTT to Kafka, the integration of Kafka into the cache as well as the Websocket endpoints and the Cache REST API have been implemented with Camel-K.

When vehicles enter or leave a zone, a zone change event is triggered. This event is made available to the respective vehicle as an MQTT message, and it is also used in the form of a cloud event to spin up serverless services and functions.

The updated zone configuration is not pushed into the vehicles, the vehicles receive the current configuration via the cache API after a zone change event. 

Kafka MirrorMaker transfers the incoming data from the regional Kafka clusters to the Kafka clusters in the central IoT cloud in order to persist an aggregated status of all locations and to enable stream analytics, for example.

Furthermore, the relevant data from the regional Kafka clusters are stored in an S3-compatible data lake in the central IoT cloud, and are then used for machine learning.

Matching with weather data

The human’s emotional response does not happen in a vacuum and is not only reacting to choices that the automated vehicle is making. The emotional responses must also be mapped to external factors outside the vehicle as well, such as  weather data.

Weather data can then be pulled in from public cloud providers, private weather data providers, or even the local weather data. This information can then be pulled together to add perspective. For example, if the weather data say that the roads are slippery, it can correlate that to the emotional data from the HDDP.

In conclusion, what does this mean in real terms?

Using the HDDP can offer a wealth of biometric data to be implemented in practical scenarios, and offer a more human-centered approach to applications at the edge of the network.

For example, in an emergency-braking situation, it is likely that automated systems will be able to engage emergency brakes without having to access cloud computing power. However, data on how that breaking action impacted those in the vehicle isn’t inherently captured by that system. The HDDP can read the passengers’ body language and process real-time data to minimize the stress the breaking action has on the passengers.

None of this would be possible without 5G connecting the edge of the network to the power of cloud computing, nor without the cloud-native platforms and applications facilitating how that data is transferred and processed.

Red Hat offers solutions to facilitate digital transformation efforts within the automotive industry:

• Red Hat OpenShift is an enterprise-ready Kubernetes container platform with full-stack automated operations to manage hybrid cloud and multicloud deployments. It is optimized to improve developer productivity, promote innovation, enhance customer connectivity, and improve communication and collaboration with partners in the automotive industry.
• Red Hat Application Services enhances solution integration across complex network infrastructures, centralizing control, automating processes, and improving connectivity.

Our hybrid cloud strategy—supported by community-powered open source technologies—brings a consistent foundation to any cloud deployment: public, private, hybrid, or multicloud. Hybrid cloud is Red Hat’s recommended strategy for architecting, developing, and operating applications across a mix of cloud environments, delivering a truly flexible technology experience with the speed, stability, and scale required for digital business transformation.

This strategy gives developers a common application environment to develop, orchestrate, and run their applications while equipping system administrators and operations teams with a common operating environment to more easily manage their infrastructure. With this consistency across environments, you can optimize and automate your IT infrastructure for business agility and innovation. Red Hat’s hybrid cloud strategy is built on the technological foundation of Red Hat Enterprise Linux®, Red Hat OpenShift, and Red Hat Ansible® Automation Platform.

Discover the benefits and features of Red Hat OpenShift or start a free trial today.

Learn more about Red Hat's interest in automotive automation with these helpful resources:

• Red Hat and General Motors collaborate to trailblaze the future of software-defined vehicles (video, 2 minutes)
• Build the future of driving with software-defined vehicles (website)
• Red Hat sets sights on delivering the first continuously certified Linux platform for road vehicles (press release)
• The new standard: Red Hat in-vehicle operating system in modern and future vehicles (blog entry)
• Automotive transformation to software-defined vehicles: Red Hat point of view and synergies with state-of-the-art IT (blog entry)