Persisting vehicle parameters using Apex.Grace message storage

Introduction

In a production vehicle, parameters must be distributed between ECUs to ensure that all ECUs have the same understanding of the vehicle configuration and user settings. Examples of parameters are steering wheel placement (e.g., left side or right side), installed sensors (e.g., trunk switch) or a user setting such as seat position.

Apex.Grace 2.2 has introduced a way to store parameters at runtime. Any type of message can be stored without limitations. The messages can be later retrieved by the same actor that stored them or any other node in the same ECU depending on the configuration.

The database for distributed parameters is normally located in a central ECU that is also responsible for distributing these parameters over the network. To make sure that these distributed parameters are available even if the central ECU is unavailable, each individual ECU can have a local storage to backup distributed parameters and/or to save ECU calibration parameters or user settings. In the future, an OEM can update these parameters over the air (OTA) and then such parameters can be distributed with the mechanism described above.

Let’s take a look at how the Apex.Grace message storage feature enables this use case. First, we consider the storage architecture and client API.

Storage architecture

The architecture of the storage is split into a service and its clients.

Storage service

Since the service must be able to allocate memory, work with any type of message, and write data to disk, it cannot be ASIL-D certified. Therefore, the service runs in an independent QM process. The default behavior of the storage service is to store data in an in-memory cache and only commit it to the persistent storage on shutdown. Users can also request an explicit commit at any time via the client API. Multiple named instances of the service can be run in parallel to achieve better isolation and horizontal scalability.

Storage clients

There are two types of client API available. These types differ in their scope of application and flexibility:

1. Strongly typed: suitable for usage by ASIL-D applications

2. Generic, dynamically typed: suitable for usage by tooling and introspection software

This means that message types that are going to be stored and retrieved must be known at compilation time for applications aiming for safety certification, but this is not required for other applications.

Client API

Structure

The API is expressed in terms of three main entities:

1. A communication object that represents a connection to an instance of the service. The communication object can be created explicitly or implicitly by other APIs

2. A database object that can roughly be thought of as a physical file

3. A data store object that represents a section of the database that stores a certain type of message

   1. The message type is the unique key of the record

   2. In case there is a need to store more than one message of the same type, an additional integer index key can be introduced alongside the message type

   3. Each data store can also keep an arbitrarily long history of the previous values that can be retrieved

   4. Saving to a data store can be done with a “transient” flag set. This makes sure that the value will never end up in the persistent storage, effectively providing “exists until the next reboot” semantics for the saved messages

Examples

The following examples are for the safety-certifiable (non-generic) API. The generic API is very similar, except for the message types, which, in the case of the generic API, are provided not as C++ types but rather as runtime strings.

Saving and fetching a strongly typed message

Note that all the “getter” APIs below have “get or create” semantics.

namespace storage = apex::storage;

// Get a database associated with the default instance of the service,
const auto db = storage::get_database("my_db");
// Get a data store of the required message type
const auto store = db->get_store<MyDataMessage>();
MyDataMessage msg;
msg.data.int64_value = 42;
store->save(msg);
// ...
// Retrieve the saved data later
MyDataMessage another_msg;
store->fetch(another_msg);

Committing and dropping entities

A database can be asked to commit the data to the persistent storage explicitly:

const auto db = storage::get_database("my_db");
const auto store = db->get_store<MyDataMessage>();
MyDataMessage msg;
msg.data.int64_value = 42;
store->save(msg);
db->commit();

A data store or the whole database can be dropped. Note that the changes will not be persisted until the next service shutdown or an explicit commit() call:

const auto db = storage::get_database("my_db");
const auto store = db->get_store<MyDataMessage>();
// Drop a store
store->drop();
// Drop the whole database with all its stores
db->drop();

Watching data stores

The service can be asked to send a push notification every time a certain data store gets updated. The price paid on the user side is one additional thread per communication object in the context of which all the registered callbacks are called:

const auto com = storage::communication::create(node);
const auto db = com->get_database("my_database");
const auto store = db->get_store<MyDataMessage>();

store->add_watch(
  [](const MyDataMessage & msg) {
        std::cout << "Value updated: " << msg.data.int64_value 
        << std::endl;
  });

com->start_watching();

MyDataMessage msg;
msg.data.int64_value = 42;
// “Value updated: 42” will be printed after this operation
store->save(msg);

CLI tool

The CLI tool allows to perform every operation that API is capable of via a command line. The data input/output format of the tool is a JSON representation of the message:

# Save a message:
cat msg.json | message_storage_cli --database my_db --type "my_msgs/msg/MyDataMessage" --save

# Read it back and print it to the console as JSON:
message_storage_cli --database my_db --type "my_msgs/msg/MyDataMessage" --fetch

Rest-API and GUI

Optionally, a Web-API service can be added to the system. The Web-API service provides RESTful access for all the functionality the C++ API provides, including watches, which are implemented via the WebSocket protocol. Apex.Grace provides an example of Web UI that utilizes this functionality in order to provide a Web-based GUI for message storage:

The users can use this example UI to easily implement more application-specific storage UIs of their own using pretty much any technology that can interact with a RESTful API.

Implementation

Let’s go back to the use case defined earlier. Due to the nature of the distributed network inside the vehicle, a single ECU can not rely on the fact that these distributed parameters are always available on the network. Therefore the following algorithm can be implemented:

1. All parameters needed by a single ECU get a default setting during development. This setting should be “safe” to run on until the distributed parameter is available on the network.

2. Once the central ECU starts to distribute parameters over the network, the single ECU mentioned above collects the needed parameters and stores them in a non-volatile memory that persists over power cycles.

3. These new parameters are then used until there is a change in parameter setting from the central ECU. A change in parameter can be due to a user of the vehicle changing a setting in the GUI or a workshop repair has reconfigured the vehicle.

This algorithm can be handled by the message storage feature, which supports the functionality of persisting and retrieving messages in runtime. The stored messages can be accessed by the same or other applications, as well as by external tooling both provided by Apex.AI and implemented by our users according to their particular needs. Message storage was designed to answer the demand expressed by many of our users and Apex.Grace itself builds upon it for future abstraction of certain patterns of usage, like views and parameters. Subscribe now to stay up to date on the latest from Apex.AI.

If you are interested in using Apex.AI products for your projects, contact us.