Executive Summary

As the complexity of automated driving functions rapidly increases, requirements for testing and development methods are growing. Testing in virtual environments offers the advantage of completely controlled and reproducible environment conditions.

The Open Simulation Interface (OSI) started as a generic data-exchange interface for the environmental perception of automated driving functions in virtual scenarios.

In tandem with packaging specifications, like the OSI Sensor Model Packaging (OSMP) specification, it provides solutions for simulation models exchange across different implementations.

As a part of the increasing adoption of OSI in the industry, and its movement to ASAM as its home, several new use cases and extensions for OSI/OSMP have been identified and are specified in this project proposal for further development and evolution of OSI and OSMP to cover the entire automated driving simulation domain.

1. Overview

1.1. Motivation

Development toward fully-automated vehicles continues to tackle various technological challenges, one of which is the verification and validation of underlying systems. For autonomous driving functions, this includes several sensors, perception algorithms, and actuators. Simulation is considered one of the key aspects in achieving the leap from automation level 2/3 to levels 4 and 5.

The current state of OSI, as a generic interface for environment perception of automated driving functions in virtual scenarios, is not entirely suitable for virtual verification and validation of automation levels 4 and 5 where more detailed models with higher fidelity results are needed. To be able to support such models, OSI should be extended to support the environment’s description on different abstraction levels.

Given the success of OSI in its field, additional use cases are coming into focus, such as whole traffic participant (aka agent) modeling, usage of OSI as a visualization interface, or for measurements, vehicle dynamics and driver modeling, modeling of V2X communication, etc.

Since OSI does not stand alone in the field of simulation of autonomous vehicles, consistent synchronization with other relevant standards, including ASAM hosted ones, like OpenDRIVE/OpenCRG, and OpenSCENARIO, as well as external ones, like ISO 23150, NDS, is necessary. While this has already been partially achieved in the past with regard to a certain (draft) version of related standards, more work is needed to ensure ongoing harmonization.

Given the expanding scope of OSI, as well as the higher degree of detail that can be transported, the area of performance and packaging will need incremental and non-incremental improvements to allow for the usage of OSI across the whole range of use cases, from real-time HiL simulations, faster than real-time object-level simulation, to slower than real-time highly detailed physics-based simulations.

Beyond the purely technical evolution of OSI and OSMP to meet current and future needs, increased emphasis needs to be placed on providing both concise reference documentation as well as comprehensive introductory and guidance documentation, to ensure that OSI is ready to be used to its fullest potential by the ever-increasing user community.

1.2. Use Cases

1.2.1. Sensor Modeling

UC 1: Abstracted Object-Based Sensor Model

Type

"Technical Use-Case"

Title

Use Case 1 : Abstracted Object-Based Sensor Model

Description

The current interface SensorView should contain more information so a stochastic based model in accordance to the paper Towards a Generally Accepted Validation Methodology for Sensor Models can be done.

Abstracted Object-Based Sensor Model

Missing road surface material properties and other static/moving objects material properties are required.

A Distinction to Use-Case 2 is that components representing the simulation environment are still on the bounding-box-level.

UC 2: Direct Geometry Access Sensor Model

Description

The aim is to extend the usage of OSI to support physics based modeling.

Direct Geometry Access Sensor Model

Starting from the left side of the figure, the "Simulation Environment" receives the osi3::SensorViewConfiguration message from the sensor model and provides as an output the SensorView** message which is an extended version of the currently available osi3::SensorView message.

In SensorView** information about more detailed 3D models and associated material properties for physics based simulation is provided. In this use case, SensorView** also extends to include other properties/parameters that are needed for a proper physics based simulation, e.g., adequate or a more detailed description of "Weather Conditions" and "Road Surface Properties".

Note: How this information is made available to the sensor model, is not in the scop of this proposal.

In the "Phenomenological Sensor Model" block, the "Currently Available" block depicts the initial set of parameters provided by the current version of osi3::SensorView message, note that this extends to include the osi3::GroundTruth data. "3D Geometries" for this use case are more detailed, i.e., a 3D model describing a vehicle would consist of sub 3D model parts describing its components, e.g. (chassis, wheels, headlamps, windows, etc..). This is necessary not only for obtaining a more detailed geometry but also for achieving the necessary level of granularity when assigning corresponding material properties, e.g., optical and electrical.

UC 3: Sampled Geometry Sensor Model

Type

"Technical Use-Case"

Title

Use Case 3 : Sampled Geometry Sensor Model

Description

In use case 3 the geometric information and all related properties (e.g. material properties) exist inside the simulation environment, and usually sensor models gain access to a sampled/processed representation of this information, e.g. through ray tracing.

Sensor Models do not have direct access to the underlying information, but are informed about it through the sampling mechanism.

In this way sensor models do not need complete, and potentially costly access to the overall scene and all of its components, but can efficiently sample the environment and model their expected behavior, i.e., provide sensor data returns,e.g. camera, radar or lidar sensor model returns.

It also enables making use of complex and optimized algorithms, e.g. ray tracing or wave propagation functionalities, from the environment simulation without duplicating all of this inside the sensor model.

As a trade-off, this also makes the sensor model behavior more dependent on the implementation used by the simulation environment.

Currently this use-case is already supported through the sensor-technology-specific SensorView sub-views. However, this should be enhanced to cover more functionality as needed by advanced sensor models.

Sampled Geometry Sensor Model

A Distinction to Use-Case 2 is that the geometry and material properties are not transferred from the simulation environment to the sensor model, but rather the sampled sub-view is only exchanged.

UC 4: Interface for Visualization

Type

"End-User Use-Case"

Title

Interface for Visualization.

Description

The focus here is on the experience of the driving simulation, to see "what is happening". This is necessary for studies but also for quick debugging on a visual level. Image-Injection to sensor models is not the use-case. It should be enabled that Graphic-Engines are exchangeable. Also the other way around, Simulation-Environments should be modular to the used Graphic-Engine. Therefore a new interface or an extension of existing formats needs to be done.

1.2.2. Traffic Participants & Infrastructure

UC 5: Studies in mockups or connecting a simulator to the simulation environment

Description

There is a need to use Mock-Ups (modified, real cars) in combination with the simulation environment for studies. Test persons take place and drive through specific scenarios. In the foreground is the interaction between the driver and the car. So far there is no possibility to transport signals like

  • Output from the Mock-Up: Steering-wheel angle, pedalry positions, gear position, …​

  • Input for the Mock-Up: Steering-wheel torque, RPM, gear, …​

The goal is to fulfil the "Mock-Up"-requirements to run a simulation.

This is an issue regarding the vehicle-/dynamics-model interface. As there are similar contents in Use-Case 10, the discussion should go on together.

UC 6: V2X Communication

Description

V2V interoperability communication systems enable information exchange among vehicles, improve mobility and reliability and enhance routing protocol by providing direction and route optimization through a network. Connected vehicles can use wireless communication to exchange sensor data, allowing them to enlarge their sensing range and improve automated driving functions, thereby maximizing information sharing among vehicles. The perception of vehicles may be improved beyond the capabilities of traditional sensors by the exchange of location data (i.e. time, position, heading and speed), vehicle dimensions, vehicle type, yaw rate, acceleration and many other vehicle parameters. This can also allow V2V (and V2X) usage to increase the vehicle’s positioning accuracy from sensors and better object recognition.

  • Standardized representations of objects to allow collaboration & exchange; objects might be also road debris

  • OSI messages might need to be more clearly structured to further support this

  • Performance considerations have to be taken into account, such as sending multiple messages in parallel

  • Security considerations and plausibility checking: An attacker could cause the receiver vehicles to inappropriately react to the faulty message content and possibly face critical consequences such as collisions (1)

A use case in an urban area serves as a good candidate where two test vehicles are used to transmit and receive data based on e.g. IEEE 802.11p standard and European Telecommunications Standards Institute (ETSI) ITS-G5 (2)

There is an ongoing standardization of the semantic interfaces in the ISO 23150, i.e. the ISO defines which sensor data or signals are mandatory or optional and how are they defined, e.g. in terms of coordinate systems and units. Earlier drafts of this standard have already been taken into account during OSI development, but ongoing changes will also be reflected due to the other sensor use cases of OSI.

Literature

(1) Sensors 2019, 19(24), Hyogon Kim and Taeho Kim3; https://doi.org/10.3390/s19245493

(2) Transp. Res. Procedia, vol. 27,Sep. 2017, S. Eckelmann et al., V2V-communication, LiDAR system and positioning sensors for future fusion algorithms in connected vehicles. https://www.sciencedirect.com/science/article/pii/S2352146517309298

Notes

ETSI has started the standardization of collective perception with the definition of the so-called Collective Perception Service (CPS) . The current version of the ETSI Technical Report defines the Collective Perception Message (CPM) and the CPM generation rules. A CPM is made of one common header and multiple containers with information about the vehicle that generates the CPM, its onboard sensors (range, field of view, etc.), and the detected objects (position, speed, size, etc.). The CPM generation rules identify when a new CPM needs to be transmitted and the information it should contain. These rules have a clear impact on the effectiveness of collective perception and on the channel load generated by the transmission of these new CPM messages.

UC 7: OSI as measurement file

Description

OSI is used as a measurement file to store in-vehicle measurement data of EGO own series production environmental detection sensors as well as reference detection sensors. The target message fields in OSI are all 'detected' messages such as detected moving object, detected traffic sign as well as detected traffic light for instance. In contrast to the simulation one must pay no attention to sensor models in the measurement use case since one is dealing with real sensors and sensor data.

Actors

ADAS/AD test and validation engineers, ADAS/AD analytics platforms to handle large amount of naturalistic driving data, ADAS/AD measurement tools and software.

Notes

A lot of driving data is generated every test day, and everybody has different names for different measurement channels. Since the amount of measurement channels are increasing, the whole surrounding environment is recorded and abstracted as a digital replica, the effort for an ADAS/AD test engineer or a data scientist to parse and understand all the data is immense. A unified way how to store EGO and environmental test data in a measurement file is therefore greatly needed.

UC 8: Provide interface for Traffic Participants

Description

OSI should be extended to cover the whole interface for a traffic participant, especially the back-interface to update simulator status with the new state of the traffic participant.

At a basic minimum, such an interface will define a model packaging OSMP model type of traffic participant, which will receive 1..n SensorView inputs, corresponding to actual or notional sensoric perception of the world state, and 1 TrafficUpdate output message, comprising the mutable parts of MovingObject, allowing the model to provide updated state information of itself to the simulation environment.

The taken approach should ensure that nesting of model types, e.g. sensor models inside a composed traffic participant model, will work correctly.

Obvious additional extensions in packaging would be the ability to package multiple participants into one model block (e.g. a traffic simulator) for efficiency reasons. Furthermore, the ability to spawn / instantiate traffic participants during runtime is seen as an important feature in order to perform efficient simulation in multi-agent environments.

Currently such approaches are already being prototyped inside the SETLevel4To5 research project, these experiences should be taken into account.

Further extensions as per Use-Case 14 should be closely coordinated.

Notes

Currently, the description of a pedestrian is not sufficient in OSI and does not belong to the MovingObject. In order to output (at the above mentioned back-interface) a more nuanced version of the pedestrian than a Bounding Box, the specification for a pedestrian needs to be detailed. This description - as a sub-message being part of OSI::GroundTruth - can also be useful for (especially camera) sensor models, which might want to be tested on pedestrian poses, gestures etc. The more detailed description might also be useful for visualizers and animation models.

UC 9: Extending OSI for driver models

Description

Integrating external driver models while protecting the subcontractor’s source code could be an important use case in the automotive industry. Therefore, the interface between the simulation framework and the driver model shall be defined and standardized. In OSI, there already exist a description of the Ground Truth and also a mechanism to filter for a predefined area around the host vehicle. In this use case the goal is to enable the concurrent control of multiple external driver models and enrich the OSI description in order for a typical driver model to be able to perform its calculations.

While this use case is expected to build upon use case 12, where the road specific information gets further detailed, an interface for control commands will be necessary as well (see use case 14). Therefore, an event based communication channel needs to be established. Because the runtime control of the events is conducted by simulation frameworks / scenario engines, the additional features in OSI shall be closely aligned with the respective tool vendors. The perspective of the driver models, most importantly, needs to be accounted for, as well. This might result in the need for data exchange at initialization phase, e.g. to allow configuration of the traffic participants / driver models and to instantiate them during runtime.

In order to drive the harmonization within the ASAM standards, the solution shall be able to support the concepts of OpenSCENARIO. Especially the assignment of external controllers – in this case driver models – needs to be guaranteed, also during runtime. Besides assignment, it needs to be specified which commands can be transferred to the driver model in particular.

Notes

Standardization of the interface from the driver model to the vehicle model seems to be difficult (see use case 10, independent of signals being routed through the simulator framework or not). The reason is that the degree of entanglement of models (especially in the area of trajectory planning and dynamic modelling) and OEM specific requirements can differ. Discussion should reveal if a standardization is useful in this regard.

UC 10: Extending OSI for vehicle models

Description

This use case describes the need for an interface which allows coupling a vehicle model to the simulation. The vehicle model primarily models the dynamics of a vehicle and its constraints. The exact need depends, if the vehicle model has access to the data structure of the environment simulation (case 1) or if it is external (case 2).

Case 1: The dynamic model receives input from the actuator control. Here, the source of input could be an AD-function, a driver model or a real driver (DiL). The input might even be predefined in the scenario description, so that the source of input would be the simulation framework or an external scenario engine. Due to different possibilities of cuts within the actuator management, different options for the connection to the driver, driver model or AD-function shall be discussed. A standardized interface to the vehicle model should respect different options in order to account for different cuts and to allow the above mentioned slightly different use cases. The output of the driver model shall be an updated state of the modelled object which is part of use case 8. Beyond that, an extension for more (perceivable) in-vehicle data shall be thought of. Here, the content might be similar to use case 4.

Case 2: Because the kinetic behavior of the vehicle model depends on the environment simulation (variables are e.g. wind, friction, …​), in this case there needs to be an additional interface from the simulation framework to the vehicle model.

1.2.3. Harmonization with OpenX

UC 11: ID Management

Description

This Use-Case describes the overall definition of ID types within the OpenX standards. Using simulation for testing driving functions with multiple agents in a complex simulation environment needs clear identification for every object.

Within this simulation, an entity is defined by multiple standards, e.g. a vehicle within OSI and within OpenSCENARIO, or a lane within OpenDRIVE and OSI. For test run observation and evaluation, a consistent ID definition is needed within every OpenX standard, which are compatible with each other. Furthermore, there should to be a recommendation for using these ID fields, e.g. the ID definition for the same entity within multiple standards should be mappable.

Example situation explaining the problematic: Evaluating a simulation built with OpenDRIVE, OpenSCENARIO and OSI from one simulation environment, objects from all three standards have to be tracked to have a full, traceable test case. For error reconstruction, you have to evaluate if a possible error comes from the simulation framework, the models, or the function. Therefore you have to evaluate every simulated object and its counterparts in every standard. The following figure shows the object definition for traffic signs for the three different standards. Currently there is no possibility to have a unique correlation between "the same" traffic sign object defined in the different standards.

ID correlation for traffic signs

The problem not only occurs for traffic signs, also for most other objects, e.g.:

lanes: OpenDRIVE defines lanes through global unique road ids, and local unity lane ids within LaneSections, OSI has a global unique id for every lane and OpenSCENARIO is referencing positions through OpenDRIVE lane identification, without any correlation to OSI.

vehicles / pedestrian: OSI MovingObject identification via uint64 unique number. Defining vehicles and pedestrian in OpenSCENARIO does not have an identifier at all, only a unique name. From IT perspective, a name is a readable description for the user. The ID is the unique identification for the system. Mixing both up, can be very confusing.

A general method should be provided to offer a clear understanding of ID handling, the meaning of the IDs within entities and can be used globally within the whole simulation. A “local unique” ID system should be avoided, because of the lack of flexibility and increasing complexity within entity identification.

Notes

Using a dedicated Identifier class in OpenDRIVE and OpenSCENARIO, as it is defined by OSI, can help to manage and encapsulate this use-case. For ID management, there should be a recommendation for common use. (e.g. Universally Unique Identifier according to RFC 4122 (https://tools.ietf.org/html/rfc4122.html)). Parameters, which are not IDs according to this definition should not be called “ID”, but get a more speaking/less confusing name.

UC 12: Exchange road specific information

Description

The goal of this use case is to enrich the description in OSI - especially OSI::GroundTruth - regarding the road network. The need for a more detailed description does primarily not arise from sensor modelling, which most of the work was dedicated to so far, but from the wish to exchange data with other external instances e.g. behavior models for traffic participants or external visualizers. Those instances are usually dependent on a more comprehensive way of modelling the road network (e.g. lanes, junctions) than it is currently realized in OSI in order to do behavioral planning. This might also require enriching the representation to include information that is not inherent in the physical network, e.g. to aid trajectory generation by models across junctions.

In the following bullet points the current shortcomings of the description language are outlined briefly, without claim to completeness. It is assumed that additional potential for improvement will be found while working on solutions in-depth.

  • For coupling of the above mentioned external instances, the road network is not well enough described in OSI, yet.

  • There are only few lane types (e.g. lanes for busses or bicycles are missing).

  • The implementation of the centerline and the corresponding rules are not ideal for modelling traffic participants and leaves some room for interpretation (e.g. when a new lane starts).

  • There are no lanes defined on junctions.

In favor of modelling traffic participants, a road and lane representation which is more related to the OpenDRIVE description, seems suitable. This is in line with ASAMs broader intentions of harmonizing the standards. Different concepts being part of OpenDRIVE can be imagined in OSI such as the representation of s-/t-coordinates, the reference line and mathematical representations within road segments. The working group implementing new attributes should consider the needs of behavior models and potentially other applications while keeping the whole concept of OSI consistent.

Notes

Comments on essential gaps:

GroundTruth/Lane/Centerline:

  • The course of the centerline is not clearly defined. In general and in special uses cases like shared lanes, lanes with varying width, construction sites, etc.

  • A clear definition for the centerline is important since it is also basis for the lane direction.

  • Maybe the centerline should not be part of GroundTruth at all since it can be considered as a sensor model output, which is based on the course of the lane boundaries.

GroundTruth/Lane:

  • How to model highway entrances and exits? Can lanes be connected directly (as some pictures in GitHub assume) or should the concept of OpenDRIVE be used (⇒ use junctions for connecting the highway parts with the on- and off-ramps)? GroundTruth/LaneBoundary:

  • OSI demands for lane boundary points at the strip-start and strip-end for dashed lines. However, one cannot decide which part of the dashed lines is the strip and which part is the gap.

UC 13: Provide map information in OSI messages

Description

Going beyond use case 12 to a degree, traffic participant models will need access to actual map and navigation data, breaking down into two sub-use cases:

  • Ego vehicle models will want access to actual HD map data to perform their internal navigation and localisation functions; while there are cases where that data is being provided to the Ego vehicle model through other means (especially for real-life-based scenarios), there are use cases where this information is expected to be provided by the simulation environment, especially for synthetic scenarios.

  • Other traffic participant models will want access to potentially simplified road network data that allows it to provide simplified navigation and trajectory planning. This will include data like likely trajectory shapes across junctions, which go beyond the data that usually comprises the actual road network data (i.e. this is - OpenDRIVE notwithstanding - not the same as actual lane shapes).

While the approach to this Use Case should take OpenDRIVE into account, it must also take other road network and map standards, as well as future evolution of OpenDRIVE into account, while acknowledging the simulation-centric OSI special needs.

UC 14: Exchange scenario specific information

Description

The OSI extensions for traffic participants according to Use-Case 8 should be extended to allow for a control channel between a scenario engine (either stand-alone or part of the simulation environment) that is tasked with scenario execution and traffic participant models, to effectuate scenarios as per the scenario description.

At a basic minimum, such an interface would be a TrafficCommand additional interface into the traffic participant models allowing the scenario engine to forward atomic actions to the traffic participant for execution. This command level should be coordinated with OpenSCENARIO 1.0 and 2.0 standards/standardization projects.

A concern to keep in mind is the distribution of work between traffic participant model and scenario engine.

Currently such approaches are already being prototyped inside the SETLevel4To5 research project, these experiences should be taken into account.

UC 15: Consistency between OSI and OpenDRIVE

Description

In use-case 12, shortcomings of OSI for the description of the road network are described and it is mentioned that the description in OpenDRIVE is sometimes more comprehensive than in OSI.

Since OpenDRIVE and OSI are often used in conjunction, with simulation models accessing information from OSI and OpenDRIVE for related entities, the consistency of OpenDRIVE and OSI needs to be guaranteed.

Obvious areas for harmonization are for example the way that traffic signs are represented in OSI and OpenDRIVE, where OpenDRIVE offers a more comprehensive approach, especially in the context of internationalization.

In other areas different representations might have to be retained, but the ability to map between the two should be ensured (e.g. classes of static objects, or lane ids).

And in other cases OSI should be enhanced to be able to carry a super-set of the required information, especially in light of the requirements of support for other road network formats, cf. use case 16 (NDS).

Analysis will have to keep the runtime requirements (including performance and abstraction levels) of OSI in mind.

UC 16: Consistency between OSI and NDS

Description

In use-case 15, the needs for harmonization of OSI and OpenDRIVE are detailed.

Since in simulation environments NDS is also used for road network information (e.g. in the SuT, but also as a source for the overall simulation road network), consistency with NDS should also be guaranteed, while adhering to consistency with OpenDRIVE.

UC 17: Uncertainties expressed as covariance

Description

Conventionally, uncertainties for multi-dimensional process variables are expressed as covariance matrices. For instance, the uncertainty of a sensed position in three dimension would be conveyed as a 3x3 covariance matrix. In its simplest case, the diagonal entries are the squares of the RMSe values for the individual dimensions.

The diagonal elements of the covariance matrix, populated from the DOFs RMS error, are comprehensible. If, however, you reported a radar detection in Cartesian coordinates, then the covariance would not be diagonal and would depend on the azimuth and range. The variance are usually only estimated or known in range and azimuth separately. So the transform into Cartesian would result in non-diagonal covariance.

Therefore, the more general case to express uncertainty of multi-dimensional process variables is a matrix. Most consumers of processed sensor data, i.e. perception outputs, e.g. motion planners would expect cavariance matrices associated with process variables.

1.3. Requirements

In the context of ASAM standards this describes requirements that the output of the project should satisfy. In general these requirements should be derived from the use cases described in this document.

The following requirements are sectioned by the corresponding Use Case.

1.3.1. Abstracted Object-Based Sensor Model

  1. On a generic level, the standard should be able to provide all necessary information for stochastic based sensor modeling.

  2. Missing information for stochastic based sensor models need to be collected, e.g.

    1. What necessary inputs for a stochastic model are not yet available in OSI?

    2. Does OSI Sensor Data covers the outputs of a stochastic model?

  3. An Extension of existing OSI messages, e.g.

    1. SensorView to include more detailed road surface properties or other material properties.

    2. Corresponding 3D geometry of osi3::MovingObject shall be represented in a more granular level, i.e., a moving object shall consist of its components separately transmitted.

      1. E.g., a vehicle can be represented as a composition of wheels, chassis, window, license plate, headlamps, etc…​ Note: the 3D geometry representing such sub-components can still be represented as 3D bounding boxes.

1.3.2. Direct Geometry Access Sensor Model

  1. On a generic level, the standard should be able to provide all necessary information for physics based sensor modeling.

  2. For osi3::SensorView** message:

    1. osi3::MovingObject message shall be extended to include additional information relevant for physics based modeling. Such information includes, but it is not limited to:

      1. Electrical properties, e.g. permittivity, permeability and conductivity.

      2. Optical properties, e.g. reflectivity.

      3. Mechanical properties, e.g. surface roughness, color.

    2. When applicable, requirements defined in 2 and 3, shall also be adapted to osi3::GroundTruth messages such as osi3::StationaryObject, osi3::TrafficSign, etc…​

    3. osi3::EnvironmentalConditions message shall be extended to meet physics based modeling and simulation pre-requisites:

      1. AmbientIllumination shall also include information about the electromagnetic spectrum.

      2. AmbientIllumination shall be extended to include information about the vehicle’s interior illumination.

      3. Fog description shall be extended to include information about its density distribution, particle size and composition. …​precipitation description shall be extended to include hail, snow, particle size and distribution.

      4. When applicable, definitions for environmental conditions from the ISO-34504 shall be taken into consideration.

  3. For osi3::SensorViewConfiguration and more specifically sensor specific (e.g., lidar, camera, radar) view configuration shall be extended to include a more detailed set of parameters e.g.:

    1. Operating electromagnetic spectrum for CameraSensorViewConfiguration.

    2. Bandwidth, polarization for RadarSensorViewConfiguration.

    3. Emitter intensity distribution for LidarSensorViewConfiguration.

1.3.3. Sampled Geometry Sensor Model

  1. Sub-SensorView messages, e.g. osi3::RadarSensorView, should be extended to meet the requirements addressed in use case 2-3.

    1. E.g., reflection as an output of osi3::LidarSensorView and osi3::RadarSensorView, may not be detailed enough to cover the outputs of a simulation environment as depicted in use case 2.

      • Corresponding to the fact that several use case may imply several OSI messages extension:

  2. Continuous alignment of OSI with "Road vehicles — Data communication between sensors and data fusion unit for automated driving functions — Logical interface" standard, ISO/DIS 23150.

  3. Entities of OpenSCENARIO (e.g. complex types such as Vehicle, Pedestrian or MiscObject or simple types such as PrecipitationType or Cloudstate; c.f. OpenSCENARIO 1.0), shall be matchable to the elements of OSI. (The harmonization from the perspective of OpenSCENARIO will be conducted within OpenSCENARIO 1.x project).

    • Corresponding to use case 4: Interface for Visualization the following requirements shall be satisfied:

  4. Missing information for visualization need to be collected

  5. An Extension of existing formats, e.g. ground-truth, needs to be done

  6. A new interface for the graphic-configuration has to be created

  7. A new interface for the control of the graphic-engine has to be created

    • Corresponding to use case 5: Studies in mockups or connecting a simulator to the simulation environment the following requirements shall be satisfied:

  8. Input from the driving simulation environment (the mock up is connected to) shall be transported to the vehicle model: Steering-wheel angle, pedalry positions, gear position, …

  9. Output from the vehicle model shall be transported to the driving simulation environment: Steering-wheel torque, RPM, gear, …

  10. Pay attention to: https://code.asam.net/simulation/standard/openscenario/issues/118

1.3.4. V2X communication

  1. All cases , V2V, V2I, V2P and V2N need to be considered

  2. An approach to handle V2X communication has to be found.

  3. It needs to be defined, if the description needs to be transport protocol agnostic or not

  4. It must be possible to reflect measures that V2X protocols employ that

    1. guarantee that messages are not modified by attackers

    2. run plausibility checks

    3. evaluate the transmission performance

    4. deal with multiple messages and

  5. It should be possible to

    1. detect observation latency

1.3.5. OSI as a measurement file

  1. More information about the EGO vehicle can be stored (e.g. brake pressure, steering wheel angle,…​)

  2. Sensor specific data can be stored

    1. Handling of large volume raw data

    2. Handling of preprocessed data (e.g. radar objects/targets,…​)

  3. Sensor specific calibration values and configuration values are supported

    1. Intrinsic and extrinsic calibration values

    2. Sensor descriptive meta data

    3. Support of geo-coordinates for localization and positioning and more geo information (e.g. related transformation, number of satellites, …​)

  4. Sensor specific health monitoring information can be stored (e.g laser lost x layer, GNSS has x satellites in sight,…​)

  5. Environmental sensor measurements can be stored (temperature, weather, lighting measurements,…​)

1.3.6. Provide interface for Traffic Participants

  1. It must be possible for a traffic participant to be controlled external to the main simulator

  2. It must be possible for an externally controlled participant to update the main simulator with its latest instantaneous state:

    1. It must be possible for the traffic participant to specify:

      1. Unique identifier for this participant

        1. This must be correlated to the identifier on the simulator ground truth

      2. Position

      3. Orientation

      4. Participant type (e.g. vehicle, pedestrian, etc.)

      5. Participant dimensions (length, width, height)

    2. It should be possible for the traffic participant to specify the following for vehicle participants:

      1. Light state (indicators, headlights, brake lights, etc.)

      2. Velocity (linear and angular)

      3. Acceleration (linear and angular)

      4. Per-wheel data:

        1. Wheel orientation (relative to the main vehicle, i.e. to indicate steering left or right)

        2. Wheel angular rotation (i.e. current rotation around the axle)

        3. Wheel rotation speed

    3. It should be possible for the traffic participant to specify the following where applicable:

      1. Participant sub-type (e.g. for a vehicle this may be "bus", or "taxi", or "sports car")

      2. An (OSI-)opaque description or identifier, e.g. to specific 3D model identifiers

  3. It should be possible for the traffic participant to optionally provide a predictive future trajectory to aid with planning for logic for simulator controlled participants:

    1. It must be possible to specify a future trajectory of arbitrary length (in time)

    2. It must be possible to specify a future trajectory for arbitrary points in time (in the future)

    3. It must be possible to include the following for each "step" in the trajectory:

      1. The time for which this step applies

      2. Position

      3. Orientation

    4. It should be possible to optionally include the following for each "step" in the trajectory:

      1. Velocity (linear and angular)

      2. Acceleration (linear and angular)

  4. It should be possible for multiple traffic participant updates (instantaneous and future trajectories) to be encoded in a single top level message

  5. It should be possible for externally controlled participants to be added and removed at runtime

  6. There should be a more granular description of pedestrians, which should be able to be updated by the TrafficParticipant model.

  7. There should be a more granular description of cyclists, which should be able to be updated by the TrafficParticipant model.

1.3.7. Extending OSI for driver models

  1. One or more Agents should be able to be packaged into one model.

  2. It should be able to apply parameters to agent models, standardizing this might be out of scope.

  3. There should be a standardized interface between driver model and the rest of the vehicle model.

  4. In this interface there should be mandatory and optional fields.

  5. Standardized definition of Input- and Output-Interfaces to make the driver models exchangeable

  6. Standard Interpretation of the Interface beyond individual companies

  7. Pay attention to: https://code.asam.net/simulation/standard/openscenario/issues/118

1.3.8. Extending OSI for vehicle models

  1. Standardized definition of Input- and Output-Interfaces to make the vehicle models exchangeable. Standard Interpretation of the Interface beyond individual companies

  2. There should be a channel from the Environment simulation to the vehicle model that enables modelling of vehicle dynamics

1.3.9. ID Management

  1. Object-IDs over different standards should be mappable (or even unified, where possible)

  2. There shall be a manageable ID definition type

  3. The ID concept must be harmonized with OpenDRIVE and OpenSCENARIO

  4. The ID shall only be used as identification of objects

  5. Every object, that can be referenced must have an ID

1.3.10. Exchange road specific information and Provide map information in OSI messages

  1. More Lane types need to be defined in order to support urban use cases.

  2. There should be a debate on principals about the definitions of the centerline and lane boundaries for sensor modeling and in order to facilitate coupling of behavior models (driver, pedestrian). The definition of the centerline (assuming the concept will be retained) needs to be more clearly defined, e.g. with new attributes or clearer description/rules:

    1. when a new lane starts / ends / widens / narrows

    2. special types such as shared lanes, lanes with varying width, construction sites

    3. Check the definition of driving direction because it seems to be ambiguous

    4. For guiding purposes (esp. driver models) there should be a centerline (typical driving route) on junctions, favorably for different vehicle sizes.

  3. More detailed definition for lane boundaries (e.g. dashed lines, handling of double lines, handling of guardrails, grass edges, …​) and description for usage conventions.

  4. The Lane description in OSI allows free connection of lanes. OpenDRIVE has the necessity to implement junctions as highway exit or entrances. It needs to be defined how to describe those exits or entrances in OSI.

1.3.11. Exchange scenario specific information

  1. A Control-message for the traffic is required as a top-level message in OSI, with harmonization of control actions to the atomic private actions of OpenSCENARIO 1.0/2.0.

  2. It needs to be examined whether additional synchronization mechanisms, beyond those present in OSI and in the packaging-specific specifications, like OSMP, are needed to allow proper interoperation in heterogenous distributed systems, such as:

    1. A time publish message is required as a top-level message in OSI to synchronize time between simulators and traffic participants.

    2. A synchronization mechanism is required to allow an entity that isn’t directly controlling the update of simulated time to hold up time, e.g. while performing additional computations.

    3. This mechanism (or a similar one) is required to control initial start-up, i.e. to ensure all entities are ready before starting a simulation (TBD whether knowledge of other entities needs / should be done out of band)

1.3.12. Consistency between OSI and OpenDRIVE

  1. Enable representation of international traffic signs in OSI, coordinated with the OpenDRIVE 1.6 and OpenDRIVE concept project approaches to traffic signs.

  2. Common definition of traffic sign composition (Pole as part of sign or not, auxillary signs, …​).

  3. Synchronize road marking types (broken, solid, …​) and configurations (solid broken, …​) with OpenDRIVE.

  4. Synchronize definition of traffic lights. Includes a common approach on how to combine the multiple single lights (green, yellow, …​) of a traffic light.

  5. Synchronize common object types between OSI and OpenDRIVE (e.g. soundBarrier).

  6. Define counterpart of the 'tunnel' attribute from OpenDRIVE in OSI.

  7. Evaluation of OpenDRIVE junction representation in OSI. Directly related to use cases 12 and 13.

1.3.13. Consistency between OSI and NDS

  1. Consistency with NDS should be examined. If necessary, adjustments within OSI should be performed.

1.3.14. Uncertainties expressed as covariance

  1. The uncertainty of all multi-dimensional parameters and process variables, such as but not limited to position, orientation, shall be represented by an optional covariance matrix.

1.4. Relations to Other Standards or Organizations

As the further development of the Open Simulation Interface is dependent of representations in other standards or activities in other projects, the interdependencies shall be mentioned here.

Other standards:

  • OpenDRIVE (map data with focus on driving simulation)

  • OpenSCENARIO (scenario description)

  • OpenCRG (road surface)

  • NDS (map data with focus on real world/ https://nds-association.org/)

  • FMI (Functional Mock-up Interface for Model Exchange and Co-Simulation/ https://fmi-standard.org/)

  • ISO-23150 (International Organization for Standardization/ https://www.iso.org/standard/74741.html)

  • 3D-geometry-standards in discussion (e.g. glTF)

  • sensor-rawdata-standards in discussion

Other projects:

  • SETLevel4to5 (simulation based development and testing of SAE-level 5 cars)

In the Project-Group these dependencies are well known and taken into account. The topic is considered especially in Working Package 4 "Harmonization with OpenX".

2. Technical Content

Provide a detailed description, how the identified usecases, features or issues (as per preceding chapter) shall be solved or implemented through the proposed project. Descriptions shall include, if applicable:

  • Components of a system, which shall be standardized

  • Features or functionality of the standardized components

  • Method of standardization

  • Potentially donated IP from member companies

  • Toplevel requirements to be considered for developing the standard

  • Statements about backward compatibility towards earlier versions of the standard

  • Improvements & benefits of the changes as compared to earlier releases of the standard

  • Assumptions

The technical content description shall be understandable for readers with an engineering background, but with no specific domain expertise. Consequently, a brief technical introduction may be needed.

2.1. Sensor Modeling

Development towards fully-automated vehicles continues to tackle various technological challenges, one of which is the verification and validation of the underlying system. For autonomous driving functions, this includes several sensors, perception algorithms and actuators. Simulation is considered one of the key aspects in achieving the leap from automation level 2/3 to levels 4 and 5.

The current state of OSI, as a generic interface for the environmental perception of automated driving functions in virtual scenarios, is not entirely suitable for virtual verification and validation of automation levels 4 and 5 where more detailed models with high fidelity results are needed. To be able to support such models, several working packages are proposed in order to update and extend OSI to support the environment’s description on different abstraction levels. Such an extension would provide a better support (but not limited to) to Stochastic Sensor Models and Physics Based Sensor Models.

2.2. Traffic Participants & Infrastructure

The topic Traffic Participants & Infrastructure envisions a progression of OSI towards the real meaning of the abbreviation Open Simulation interface. The implementation of OSI - to date of this proposal - is limited to an Open Sensor interface, which allows the one-directional data flow from the simulator framework over the sensor model(s) towards the AD-function.

The loop to the simulator framework via OSI shall be closed through the development within this project. The very basic idea is to be able to send the updated status of the modelled Traffic Participant back to the simulator framework. This concept shall not be limited to the System under Test (e.g. AD-function of the host vehicle), but shall be deployable for any Traffic Participant. The term Infrastructure points towards the need of those modelled Traffic Participants to receive more information about the virtual world than currently possible and also, optionally, some guidance.

2.3. Performance & Packaging

Given the increasing scope of OSI, as well as the higher degree of detail that can be transported, the area of performance and packaging will need incremental and non-incremental improvements to allow the use of OSI across the whole range of use cases, from real-time HiL simulations, faster than real-time object-level simulation, to slower than real-time highly detailed physics-based simulations.

This entails a close look at the encoding layer, currently Google protocol buffers, and potential replacments/additons, like Google Flatbuffers. Guidance on the modeling inside OSI and proper use of OSI for performance critical work will be developed.

It will also include work on making control of the amount of data being transmitted part of the standard, either through the use of dynamic field selection, separation of dynamic from static data, delta-encoding mechanisms or any other mechanisms needed to taylor and reduce the amount of data being transmitted.

2.4. Harmonization with OpenX

Since OSI is a new standard within ASAM’s Simulation domain, the harmonization with OpenX (e.g. OpenDRIVE, OpenSCENARIO) is very crucial for the interplay of the standards. The close semantic connection between OSI and the OpenX standards exists, because OSI reflects/describes the (virtual) world - in other words the Ground Truth - in a simulation-centric way whereas the OpenX standards describe it prior to simulation execution. While OSI defines the relevant information for the communication between simulation components / models during simulation execution, the information shall be relatable and consistent to the descriptive or declarative language of the OpenX formats.

3. Project Resources

3.1. Work Packages

A breakdown of the project into individual work packages and the corresponding effort required to complete them. Effort should be given in man-days.

3.1.1. Sensor Modeling

WP 1: MovingObject Message Extension

Description

Geometry and material properties extension for MovingObject

Deliverable

  • A more granular 3D geometry representation of moving objects (e.g. bounding boxes for wheels, chassis, car body etc. )

  • OSI place holders for material properties such as electrical, optical and mechanical properties.

  • Alignment, when applicable, with previously existing OpenSCENARIO entities that would correspond to moving objects.

Effort (Man-days)

38

WP 2: StationaryObject Message Extension

Description

Geometry and material properties for StationaryObject

Deliverable

  • A more granular 3D geometry representation for stationary objects (e.g. bounding boxes for windows, concrete blocks, etc. )

  • OSI place holders for material properties such as electrical, optical and mechanical properties.

  • Alignment, when applicable, with previously existing OpenSCENARIO entities that would correspond to stationary objects.

Effort (Man-days)

27

WP 3: Message Extension for Remaining GroundTruth Components

Description

Geometry and material properties extension for remaining GroundTruth

Deliverable

  • A more granular 3D geometry representation for remaining ground truth data, e.g. TrafficSign, and TrafficLight.

  • OSI place holders for material properties such as electrical, optical and mechanical properties of remaining ground truth data, e.g.:

    • TrafficSign

    • TrafficLight

    • RoadMarking…​

  • Alignment, when applicable, with previously existing OpenSCENARIO entities that would correspond to remaining ground truth components.

Effort (Man-days)

31

WP 4: Extension for EnvironmentalConditions

Description

Extension for EnvironmentalConditions

  • In the process of extending the environmental conditions, when applicable, previously defined entities in ISO-34504 will be taken into consideration.

Deliverable

  • A more precise description of various weather conditions as well as ambient illumination inside the vehicle.

  • A report clarifying the considered/not considered ISO-34504 entities for OSI’s environmental conditions extension.

Effort (Man-days)

27

WP 5: Extension for SensorViewConfiguration

Description

Extension for SensorViewConfiguration

Deliverable

A more detailed parameters set for physics based sensor configuration.

Effort (Man-days)

21

WP 6: Extension for SensorView

Description

Extension for SensorView to better support outputs from simulation environments. Such an extension may also be used to support the outputs of physics based simulation.

Deliverable

Extended Sub-SensorView messages to better support the exchange of sampled sub-vew messages (e.g. osi3::LidarSensorView) between simulation environments and sensor models.

Effort (Man-days)

21

WP 7: Create an interface for visualization

Description

Adapt existing interfaces or create new one regarding the information that is needed to do a decoupled visualization.

Deliverable

  • Extension of existing interfaces

  • Creation of new interfaces

  • Have a proof-of-concept

Effort (Man-days)

30

WP 8: Adapt/Extend the osi-validator

Description

Adapt/Extend the osi-validator regarding the new content developed in this project context.

Deliverable

Effort (Man-days)

15

WP 9: OSI Alignment with ISO 23150

Description

OSI is the “virtual” equivalence of the ISO/DIS 23150 that describes Data communication between sensors and data fusion units for automated driving functions. The aim is to ensure that OSI is aligned with ISO 23150 (where applicable) thus ensuring a correlation between virtual and real sensor data.

Deliverable

  • Alignment of OSI SensorData with ISO 23150 that may include creating new messages or updating existing ones.

  • Clarification how to adapt OSI regarding ISO23150 correctly

  • A coverage report of the OSI-ISO compatibility.

Effort (Man-days)

20

3.1.2. Traffic Participants & Infrastructure

WP 10: Define an interface from the TrafficParticpant to the simulator framework

Description

As a basis OSI::MovingObject can be choosen. A more granular 3D geometry for MovingObjects is necessary (see WP1). Beyond that, information about the TrafficParticipants' perceivable status that gets updated during simulation needs to be added (also for pedestrians and cyclists).

Deliverable

  • New message (e.g. OSI::TrafficUpdate) including content of WP1 + further attributes that are perceivable (e.g. visually / acustically)

  • A more granular description for vehicles, pedestrians and cyclists

  • If possible future trajectories

Effort (Man-days)

15

WP 11: Enable control of TrafficParticipants

Description

Develop an interface to allow event-based, external control of the Traffic Participant.

Deliverable

  • Packaging options to add and remove TrafficParticipants during runtime

  • Definition of messages for event-based communication with TrafficParticipants to provide options to guide traffic participants (esp. due to atomic Actions from OpenSCENARIO)

  • If useful for agent modelleres provide more high-level maneuver actions

Effort (Man-days)

15

WP 12: Enable V2X communication

Description

V2X Communication is not yet implemented in OSI but can reflect another input source for TrafficParticipants and should be modelled.

Deliverable

  • A configurable approach to communicate V2X data via OSI so that the intended TrafficParticipant can consume the data

  • Data and metadata of the communication need to be defined

Effort (Man-days)

30

WP 13: OSI as a measurement file

Description

OSI can be used as a measurment file format for ADAS/AD measurements

Deliverable

Examine the use of OSI for measurement reasons by considering the requirements as well as other relevant measurement standards that already exist, like e.g. ASAM MDF.

If an extension to OSI is considered useful, the standard needs to be extended to enable capturing real world measuremnt data for

  • EGO vehicle signals and descriptive meta data

  • Environmental sensor info

  • Sensor specific data

  • Sensor specific calibration, configuration data descriptive meta data

  • Sensor specific monitoring data

  • Geo data measurements

If the use of OSI in combination with a pre-existing measurement file format is found to be of use, the relevant extensions to both OSI and the other file format(s) need to be defined.

Effort (Man-days)

35

WP 14: Extending OSI for agent models (e.g. driver models, pedestrian models)

Description

This WP deals on the one hand with the packaging and configuration of agent models, while the extension of necessary road and map specific data should be closely aligned with Road-specific / map information. On the other hand it tries to accomplish the definition of an interface from the driver model to the rest of the vehicle model. Additionally, it considers further extensions for connecting a simulator to the simulation environment.

Deliverable

  • An option to package multiple agents into one model

  • Enabling the application of parameters to the driver model

  • If possible, standardized definition of the interface from driver model to the rest of the vehicle model with mandatory and optional fields

  • Proof-of-concept working with these interface(s)

  • Extensions to the driver model to allow for connection of a simulator or mockup to the simulation environment

Effort (Man-days)

45

WP 15: Extending OSI for vehicle models

Description

Beyond the connection between driver model and vehicle model (see WP16) this WP deals with the connection between environment model and vehicle model. Additionally, it considers further extensions for connecting a simulator to the simulation environment.

Deliverable

  • Standardized definition of the interface from the vehicle model to the simulation environment/ driver model with mandatory and optional fields

  • proof-of-concept working with these interface(s)

  • Extensions to the vehicle model to allow for connection of a simulator or mockup to the simulation environment

Effort (Man-days)

35

3.1.3. Harmonization with OpenX

WP 16: ID-Management

Description

Uniform and unique ID management for all OpenX standards

Deliverable

  • Evaluation of state-of-the-art unique identifier

  • Consultation on all standards

  • Overarching utilization in all OpenX

  • Integration and extension of the selected ID

Effort (Man-days)

20 (from the perspective of OSI)

WP 17: Road-specific / map information

Description

Enrich the road network description to resolve existing ambiguities and facilitate coupling of TrafficParticipants (e.g. Driver models, pedestrian models)

Deliverable

  • New lane types (in accordance to OpenDRIVE)

  • Clearer definition of centerline & lane boundaries including new rules if necessary

  • Guiding aid for TrafficParticipants on junctions

  • Eventually further topics discovered during the duration of the project

Effort (Man-days)

30

WP 18: Consistency with OpenDRIVE

Description

Extend/Adapt OSI to make sure that content in both standards is defined consistently. This work package includes a close collaboration with OpenDRIVE concept or standardization projects.

Deliverable

Each of the following entities have to be analyzed if an adaption/extension of the OSI standard is necessary, or if a mapping documentation between OSI and OpenDRIVE is sufficient.

  • Traffic signs (Also considering international signs approach from OpenDRIVE Concept project)

  • Road markings

  • Traffic lights

  • (Common) Object types

  • Junction representation

  • Potentially further entities

Effort (Man-days)

40

WP 19: Analysis of the OSI-NDS Workflow

Description

There are multiple use cases that require a direct interaction with NDS. This WP will analyse these interactions and identify currently existing gaps that might be filled in a future OSI release.

Deliverable

  • Documentation detailing the NDS/OSI interactions and recommendations on future additions to OSI

Effort (Man-days)

10

3.1.4. Performance & Packaging

WP 20: Encoding and Data Modeling

Description

The current performance levels achieved are sufficient for the initial "standard" use cases originally envisioned, e.g. object-based sensor modeling.

However it is already apparent that increased simulation complexity will place significantly increased performance requirements on OSI. This will be due to:

  • Increased quantity or complexity of sensors in simulation

  • Increased resolution of environment model

  • Increasing additions to the OSI data model

  • Completely new or wider use cases, like use for visualization, driving dynamics, etc.

  • Increasing use in real-time and faster-than real-time simulations

Currently Google Protocol Buffers are used both as the data description language (DDL), and as the encoding layer. Protocol Buffers are not tuned towards the encoding/decoding of large data quantities in a CPU-efficient manner and is thus a performance bottleneck at the encoding layer. An alternative encoding is the Google Flatbuffers encoding, which is significantly faster for encoding and does not even require decoding in all cases. It also allows selective decoding and in-place mutation of encoded data, which are especially useful in the OSI context. Since the Flatbuffers encoding is compatible with the currently used subset of Protocol Buffers used as the OSI DDL, this encoding can be used as a direct replacement, without any changes to the DDL definitions.

Given that the focus of the Flatbuffers API is on efficient encoding, it follows a different programming model than the Protocol Buffers API. The example code should be adapted accordingly, and recommendations for the use and support of Flatbuffers encoding should be established.

Additionally, a survey of the existing data definitions should be performed to ensure that data modeling takes account of the performance characterstics of both Protocol Buffers and Flatbuffers encodings, as far as possible, while still achieving other overriding objectives, like modularity, ease of use, etc.

Deliverable

  • Added Flatbuffer support in the OSI code and documentation

  • A usage guide for OSI that addresses relevant performance issues:

    • Encoding recommendations (Protobuf, Flatbuffer, etc.)

    • Performance pitfalls and recommendations

    • Interaction between data and transport layer (OSI vs. OSI packaging)

  • Relevant changes to the data definitions to improve performance in terms of data modeling

Effort (Man-days)

50

WP 21: Data Traffic Reduction

Description

Given the trajectory of further development and enhancements of OSI, performance pressure will likely increase due to the optional amounts of data that can be transported in various messages to support the underlying use cases (see also Encoding and Data Modeling). Currently OSI strives to keep data together thematically even if it has differences in change frequency, and OSI tries have as little configuration effort as possible. In the face of the currently planned evolution, this approach might no longer be feasible. Therefore this WP should examine different ways to achieve data traffic reductions in order to support larger and more complex use-cases. Among the techniques to be examined are:

  1. The configurable selection of data to be included in OSI messages

  2. The separation of static and dynamic information (e.g. in groundTruth)

  3. Use of delta-encoding techniques, either at the encoding or at the application/protocol level

While examining these potential techniques and others, the requirements of the various use-cases shall be taken into account, especially those most impacted, like e.g. the use of OSI in measurement formats.

Deliverable

  • Change proposals for the existing messages to support the selected data-traffic reduction techniques

  • Guidelines on the formatting and structuring of messages to support OSI as measurement file

Effort (Man-days)

50

3.2. Company Commitments

Member companies contribute resources for the project as per the following table.

Table 1. Company Commitments
Company Location Commitment Participant(s) Participant Email

51WORLD High Technology Co.Ltd.

China

25

Anchun Zhang; Shiqiang Bao

zhanganchun@51hitech.com; baoshiqiang@51hitech.com

Advanced Data Controls Corp.

Japan

30

Yasuyuki Nakanishi; Masahiko Kohda

nakanisi@adac.co.jp; kohda@adac.co.jp

Automotive Safety Technologies GmbH

Germany

25

Radhakrishna Kothamasu

radhakrishna.kothamasu@astech-auto.de

Ansys Germany GmbH

Germany

50

Kmeid Saad;

kmeid.saad@ansys.com

AVL List GmbH

Germany

25

Thomas Schloemicher; Jonas Ruebsam; Klaus Schuch; Ilona Cieslik

Thomas.Schloemicher@avl.com; Jonas.Ruebsam@avl.com; klaus.schuch@avl.com; Ilona.Cieslik@avl.com

BMW AG

Germany

50

Thomas Nader; Clemens Habedank

thomas.nader@bmw.de; Clemens.Habedank@partner.bmw.de

Connected Places Catapult

UK

25

Zeyn Saigol

zeyn.saigol@cp.catapult.org.uk

Continental AG

Germany

25

Rainer Aue

rainer.aue@continental-corporation.com

dSPACE GmbH

Germany

25

Hendrik Amelunxen

HAmelunxen@dspace.de

FiveAI

UK

25

Caspar de Haes; Iain Whiteside

caspar.dehaes@five.ai; iain.whiteside@five.ai

FZI Research Centre on Information Technology

Germany

25

Markus Lemmer; Sebastian Reiter

lemmer@fzi.de; sreiter@fzi.de

Huawei

China

25

Jianqin Liu

liujianqin@huawei.com

IKA RWTH Aachen University

Germany

25

Daniel Becker

daniel.becker@ika.rwth-aachen.de

LiangDao GmbH

Germany

25

Xiaotong Liu

xiaotong.liu@liangdao.de

Metamoto

USA

25

Mark Hary

Mark.hary@metamoto.com

Mitsubishi Precision Co. Ltd

Japan

25

Atsushi Araki

aaraki@mpcnet.co.jp

MMI RWTH Aachen University

Germany

10

Jörn Thieling

Thieling@mmi.rwth-aachen.de

PMSF IT Consulting

Germany

30

Pierre Mai; Thomas Sedlmayer

pmai@pmsf.eu; tsedlmayer@pmsfit.de

RA Consulting GmbH

Germany

10

Frank Hantschel; Thomas Kotschenreuther; Armin Rupalla

F.Hantschel@rac.de; t.kotschenreuther@rac.de; a.rupalla@rac.de

Tencent

China

25

hainingdu

hainingdu@tencent.com

Technische Hochschule Ingolstadt

Germany

25

Thomas Hempen; Georg Seifert

Thomas.Hempen@carissma.eu; Georg.Seifert@carissma.eu

Technische Universität Darmstadt

Germany

25

Philipp Rosenberger

philipp.rosenberger@tu-darmstadt.de

University of Applied Sciences

Germany

25

Prof. Dr. S. Schneider

stefan-alexander.schneider@hs-kempten.de

Total

605

The following intellectual property will be transferred from member companies to ASAM:

Table 2. Intellectual Property
Company (Name, Location) IP Description Value (Euros)

3.3. Effort Summary

Table 3. Required Effort (man-days)
WP Category Project Members Service Provider Total

Harmonization with OpenX

90

10

100

Sensor Modeling

195

35

230

Traffic Participants & Infrastructure

205

40

245

Performance & Packaging

75

15

90

Total

565

100

665

Table 4. Resource Check
Project Members Service Provider Total

Required

565

100

665

Committed

605

3.3.1. Budget

This section details the budget required by the project to e.g. pay service providers and the funds to be provided by ASAM. The limits are determined as a factor of the total required work effort. The corresponding effort is allocated a fixed price of 700 EURO per man-day.

Table 5. Funding limits

Project Type

Factor

New, major, minor or revision standard development project

0.25

Study project

0.25

Concept project

0.75

Table 6. Funds required for Service Providers
Task Description Effort
[man-days]
Cost
[€700 / day]

Documentation

100

70.000 EUR

Total

100

70.000 EUR

4. Project Plan

4.1. Timeline

The work packages shall be carried out as per the following time schedule:

timeline

4.2. Deliverables

At the end of the project, the project group will hand over the following deliverables to ASAM:

4.2.1. WP Sensor Modeling

  • Extended OSI messages (e.g. SensorView) to support advanced Stochastic Sensor Models

  • Extended OSI messages (e.g. SensorView, SensorViewConfiguration) to support Physics Based Sensor Models

  • Extended OSI messages (e.g. Sensor Specific-SensorView, SensorViewConfiguration) to better benefit from already existing capabilities (e.g. ray tracing) of simulation environments and to directly export pre-sampled data for sensor models to consume.

4.2.2. WP PP-1 Encoding and Data Modeling

  • Added Flatbuffer support in the OSI code and documentation

  • Usage guide for OSI that addresses relevant performance issues

  • Relevant changes to the data definitions to improve performance in terms of data modeling

4.2.3. WP PP-2 Data Traffic Reduction

  • Change proposals for the existing messages to support the selected data-traffic reduction techniques

  • Guidelines on the formatting and structuring of messages to support use case 3 (measurement file)

4.3. Review Process

The following quality assurance measures shall be carried out by the project:

  • Project member review

  • Public review

  • Reference implementation

Appendix A: Additional Resources

A.1. Performance Use Cases

This chapter describes specific usecases for benchmarking that lead to bad performace. This opens discussion and makes the problem more understandable, makes assumptions clearer and also defines specific performance problems because users will run in all kinds of performance problems and they would have multiple levers how to solve their own problems. It also serves as a basis for defining the principal components in a way which makes sense for the usecases. The usecases will serve as basis for performance testing.

WP 22: Driving dynamic simulation

Description

A scenario without stationary objects. Some people will just simulate the vehicle dynamics/physics wich in turn do not need the static objects. But on other hand need a very specific definition of the road on witch the scenario is mapped. For example the road structure at this point is completely flat, but the roads have elevation, pitch and roll properties that define a surface of the road. This should not be defined as static object since it is a derivable element. This leads to the problem of the garages in the stationary objects and parking lots. The derivable elements can be a part of a stationary objects but not vise versa. So static object and all static infrastructure can be disabled but the road and driving environment should be kept.

Actors

Developers who need precise focus on specific task like the vecile dynamics. The do not need to render every point of lane boundaries.

Notes

Reduces overhead and so increases performance which can lead to near real time performance.

WP 23: Configurations of interactive and non interactive infrastructure

Description

The road side or traffic regulation infrastructure of the future is going to be interactive. Not all need the definitions of state and communication in OSI. Those definitions will be very heavy and the definitions will make the scenarios very heavy. Performance sufficient for the initial "standard" use case, i.e. object based modeling Increased simulation complexity will place significantly increased performance requirements on OSI. This will be due to e.g.: * Quantity or complexity of sensors in Simulation * Increased resolution of environment model * Increasing additions to the OSI data model

Actors

Developers who need precise focus on specific tasks.

Notes

Reduces overhead and so increases performance which can lead to near real time performance. A configuration file which defines which fields are relevant to use and which can be ignored should be implemented.

Steps for process should be:

1. Definition minimal setup

2. Definition of pricipal compenets of OSI messages

3. Define preprocess/configuration file for needed messages and not needed messages

4. Benchmark/Test with and without preprocess

WP 24: Data modeling

Description

There are general problems in the definition of OSI data models which need to be addressed due to inefficient usage of data structures.

Examples:

- There is to much use of int32 instead of sint32 or uint32, which gives a much more efficient format for transporting. https://developers.google.com/protocol-buffers/docs/encoding

- The copyfrom function creates a problematic situation because it is type agnostic wich should be avoided. If the fields a added as variables and not as messages then there is no copyfrom function, so the message construction is more efficient.

- The 0 In the id should be used as a indicator that the message is empty. This will make serialization faster and the transmitted messages smaller.

- Dense nesting of messages

Actors

Developers which need to deal with appropriate data structures.

Notes

Enable effienct compiliation of OSI due to right usage of data strucuture and data modeling which can lead to near real time performance.

Examples of problems:

- Type of fields definition (to much use of int32)

- Avoidance of redundant messages

- Unnecessary boolean messages

- Testing and benchmarking needs to be done in each programming language (for starters Python and C++)

- Some message are to capsulated defined

- Flat is better then nested. (Effect on performance unknown)

- Ease development process for faster development

- Rules for avoiding to much nested messages which may take up much space in future

- More usage of repeated fields if element are unknown

WP 25: Subscription model for OSI
Description Currently OSI messages encode all the available data into one message although the message fields stay the same thus creating unnecessary overhead. It employs a pull model which should be replaced by a subscription model to account for performance.

Actors

Developers who need only fields to update which only change.

Notes

Reduce overhead send over the wire. Static and dynamic defintion of OSI message needs to be flagged or defined. A mechanism needs to be developed to send only the diffs of messages (delta encoding). Separation of static and dynamic information from GroundTruth.

Steps for subscription:

1. Create OSI message scenarios files with delat informations (diffs)

2. Define preprocess file for preprocssing messages before serialization

3. Benchmark/Test with and without subscription for proof of concept

WP 26: Encoding/Decoding

Description

Protobuf has problems with encoding/decoding large data quantities and is thus a performance bottleneck. An alternative is Flatbuffer, which is significantly faster for encoding and does not even require decoding in all cases. Its focus is more on efficiency, thus programming complexity is greater. Could be directly supported in addition to Protobufto be performed or give a descriptive title.

Actors

Developers who need near real time performace.

Notes

Reduce overhead send over the wire.