2_technical-content.adoc 14.5 KB
Newer Older
1 2 3 4 5
ifndef::imagesdir[] 
:imagesdir: ../images
endif::imagesdir[]


Benjamin Engel's avatar
Benjamin Engel committed
6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
Based on the considerations of the concept group, OpenSCENARIO 2.0 is proposed to be founded on the concept of a domain-specific language (see Methodology and Language Concepts chapters in the concept document), that should support all levels of scenario description, from the very abstract to the very concrete in a suitable way.

The OpenSCENARIO 2.0 concepts take proven features and capabilities of OpenSCENARIO 1.0, like its event-based scenario execution model, and place them in a more general and expressive language framework, to serve as the foundation for both incremental improvements and more revolutionary enhancements.

In comparison to OpenSCENARIO 1.0, a more detailed set of actions and attributes for the relevant simulation models shall be defined to allow for a more comprehensive scenario description and to improve exchangeability. This is addressed by the introduction of a domain model (see Concept document chapter: Architecture).

Beyond such improvements, the proposed domain-specific language supports the creation of more complex scenarios through flexible composition and parametrization of component scenarios in ways that allow realistic, emergent behavior to occur. This is expected to enable massive testing and verification of new, complex hardware and software systems, and their interaction with the complex environment of driving.

The concept group believes that the proposed concepts and direction will enable OpenSCENARIO 2.0 to supply a step-function in its ability to describe full, complex simulations and/or tests, addressing all the required elements, and utilizing existing building blocks (like OpenDRIVE or OpenCRG).

As stated above, the foundational concept of OpenSCENARIO 2.0 is to establish a domain specific language of a declarative nature. A declarative language describes what should happen on scenario execution (including the required parameterization/variation), rather than how to do it. A declarative language can also have a _dual interpretation_, i.e. provide a single scenario description which can be used to describe both how to make it and how to monitor that it _indeed_ happened. This is important if we want to condition some operation on the fact that some other scenario has happened before, without having to describe in detail _how_ to cause that scenario.


The Concept document is using M-SDL language to demonstrate all the concept. It is expected to examine if this language can serve as the foundation of the standards.

Provide a detailed description, how the identified usecases, features or issues (as per preceding chapter) shall be solved or implemented through the proposed project. Descriptions shall include, if applicable:

* Components of a system, which shall be standardized
* Features or functionality of the standardized components
* Method of standardization
* Potentially donated IP from member companies
* Toplevel requirements to be considered for developing the standard
* Statements about backwardcompatibility towards earlier versions of the standard
* Improvements & benefits of the changes as compared to earlier releases of the standard
* Assumptions

32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68
The technical content description shall be understandable for readers with an engineering background, but with no specific domainexpertise. Consequently, a brief technical introduction may be needed.

==== Domain Model

===== Entity Definition 
Responsibles for filling this sub-chapter: J. Tscheak (maybe), M. Kluge, J.Kaths, F. Bock (maybe), S. Rosenberg (maybe)
WP DM-Transfer1x - Rework Entity Definition from Concept project with regard to OSC 1.0
overhaul entity definition with regard to OSC 1.0 (maybe Jupp Tscheak, Michael, Jakob)
    * identify deviations
    * clarify if deviations are necessary or should
    * clarify naming / terminology
    * create a mapping where necessary

WP DM-Extend2x - Extend Entity Definition from Concept project and from WP DM-Transfer1x towards OSC 2.0
identify entities and their actions / methods that are introduced by high-level description of OSC 2.0 (so not part of OSC 1.0) (maybe Florian Bock, Michael Kluge, Jakob Kaths, maybe Sharon Rosenberg)
    * identify new entities, their actions and attributes that are necessary to cover high-level scenario descriptions in OSC 2.0 (e.g. Road, drive(), etc.)
    * integrate them properly in Domain Model
    * close interrelation on working-level with language group (was a lack during concept phase)

===== Extensibility 
Responsibles for filling this sub-chapter: S. Rosenberg, H. Bothe (maybe)
WP DM-Extensibility - Develop technical concept for extensibility of Domain Model
    * check with OTX where there are working groups for extensions (more firm extensions in line with Michael's (dSPACE) proposal)
    * feature sets may add parts to the domain model
    * "quick" extensions by (even single) users need to be possible as well
    * possibly introduce mandatory / optional
    * align with WP modules, libraries, etc.

===== Ontology
Responsibles for filling this sub-chapter: F. Sanchez, I. Whiteside, P. Parekh, S. Kraines (maybe), M. Nieto (maybe)
WP DM-Ontology - Develop ontology to establish relations between entities
    * synchronize with global ontology developed by ASAM
    * what are the technical artifacts that come out of the ontology development
    * Fran (FiveAI) can contribute, FiveAI developed own ontology for scenario description
    * Iain Whiteside (FiveAI) can provide a presentation on practical usage of their ontology
    * UK has developed a standard (or a PAS really) called 1883 for a driving ontology (Siddhartha could help here)
    * possible input as well from Steven and Florian Bock (Audi)
69 70 71

==== Language Concepts
 
72 73 74 75 76 77 78 79 80
ASAM OpenSCENARIO 2.0 is based on a DSL – Domain Specific Language of  declarative nature. This language demonstrates characteristics similar to a programming language, however as this is a declarative language, it specified what should happen, rather than how it should happen. The chapter described the syntax and semantics of the language as well capabilities such as libraries, and the abilities to measure and define success criteria. The language  will be designed to deliver key concepts and requirement: 
 
Readability::  The quality of being easy to be read and understand by a target audience, which includes not only domain experts, programmers and engineers, but also safety engineers, regulators, and potentially even the general public. While this has clear implications for suitable surface syntax of the language, it also requires the semantics of the language to be well-defined, regular, and focused on the problem domain, and not technical implementation artifacts.
Expressivity::  The ability of the language to allow the concise and direct expression of domain subject matter . This interacts with readability and the declarative nature of the language, but also enables for example the ability to query databases of scenarios using language semantics rather than only through meta-data-based queries.
Composability:: The quality of being able to use standard language building blocks to incrementally build up more complex behavior in well-defined ways. This requires clear rules for composition and composition operators, that afford some level of predictable behavior post composition, while also allowing for interesting emergent behavior to occur, to ensure coverage of overall realistic traffic behavior.
Portability:: The quality of being able to use the language and scenarios written in it across many execution platforms (including simulation platforms as well as real-world test tracks), without undue adjustments needing to be made. This includes the provision of suitable fallback mechanisms to deal with necessary differences between platforms.
Support for Reuse::  The quality of being able to reuse scenarios and parts of scenarios across multiple levels of abstraction, platforms, and use cases without undue adjustments needing to be made. This comprises suitable library concepts and mechanisms in the language, as well as
Extendibility::   The quality of allowing for easy extension of the underlying language mechanisms and domain model towards new requirements, including new traffic participants, their models and properties, compositions, and constraints.

81 82
===== Semantics, syntax and structure

83 84 85 86 87
This section will describe the overall structure of OpenScenario 2.0 programs, including partition into modules (files), the mechanism for module inclusion and related syntactic aspects such as defining comments, scoping (defined by indentation), character set and so on.
 
Language type definitions will include scalars, strings, enumerations, structs, actors, scenarios and modifiers, as well as their properties and members. Language constructs shall be described with their concrete syntax, their semantics, their relation to other constructs and any limitations on their usage. Where appropriate, formal denotational semantics will be provided to eliminate ambiguity.
 
Inheritance mechanisms that apply to structs, actors and scenarios shall be clearly outlined. Operators for numeric expressions, Boolean expressions, strings and other types will be listed, along with their precedence. Composition rules for scenarios and modifiers, including argument passing mechanism shall be defined. The syntax of constraints and their implied semantics will be described where applicable. Declaration and invocation of external functions, including argument passing will be defined as well.
88 89 90 91

Finally, the overall semantics of the language should be clearly and unambigously defined. In order to achieve this, it is important to define how behavior is specified by combining actors, actions, modifiers and events to atomic phases and how possible semantic unambiguities are resolved. This also includes how conditions can be used to define events (e.g. raising/falling edge, etc) and which expressiveness we allow (compared e.g. to temporal logic languages).

Also included is the definition of the exact semantics of temporal composition operators like serial, parallel and mix. Another important aspect is how to combine the two different approaches of specifying behavior: a) the declarative way (like intended for OpenScenario 2.0) and b) the more execution/command-oriented way of OpenScenario 1.0 (that still should be supported)."
92

93 94 95
===== Considerations for 1.x
===== Library Concepts and Packaging (scenario definition & scenario execution)

96 97 98 99 100 101 102 103 104 105
===== Measurement and Success Criteria
This section will outline the OpenScenario 2.0 features used to collect and classify information during scenario execution. The collected information will be used to evaluate the quality and extent of the verification process, as well as the performance of the "ego" actor.
 
These data collection features may include coverage, tracing and recording of Key Performance Indicators as well as similar mechanisms. Semantics, like defining what expressions are sampled, in what resolution and temporal aspects of sampling will be unambiguously defined.
 
Language properties for grading and classifying collected information will be described. These include filtering values, grouping them into ranges (buckets), assigning minimal and maximal values, setting goals and so on. 
 
The section will also describe language constructs for checking. These include assertions over Boolean and temporal properties, expressed using Boolean expressions and compound scenarios. Mechanisms for handling illegal values and assertion violations shall be provided, allowing the classification of errors (distinguishing between scenario failures, ego performance issues, ego failures and so on).
 
 
106
===== Siddartha's Paragraphs
107 108


109 110 111 112 113 114
==== Usage and Pragmatics

===== Test definition
How to test scenarios?
Clarification of boundaries test cases and scenarios
Outputs:
115
- Workflow guidelines (e.g. when integrated with a scenario vs. defined separately)
116 117 118 119 120 121 122 123 124 125 126 127 128 129

In this section we'll try to detail *how a scenario should be tested*.

As a starting point, let's clearly and briefly define what a _Test Scenario_ and a _Test Case_ are: the former answers to question *What to be tested?*, while the latter to *How to be tested?*.

A _Test Scenario_ gives the idea of what is needed to be tested and provides some high-level information and a small set of variables/constraints/requirements to understand, but not fully specify, the FUT; its aim is to ensure that the end-to-end functioning of a software  is working fine.
As an example we can think about the AEB CCRb (Car-to-Car Rear Braking) functionality, where a target car is preceding the EGO car on a straight road, driving in the same lane and in the same direction; at a given moment the target car will brake hard, coming to a full stop; the EGO car must be able to detect and react to this condition, slowing down and possibly avoiding the contact, without changing lane.
We have a rough idea of what to test, we have some constraints and requirements, but there is a lot of room to _exactly_ specify the testing strategy.

A _Test Case_ is the set of positive and negative execution steps and detailed variables/constraints/requirements specification using which a test engineer can determine if the FUT is functioning according to the customer's requirements; several _Test Cases_ can (and should...) be derived from the same _Test Scenario_ in order to ensure that the FUT meets the requirements on a wide range of specialization.
Back to our AEB CCRb example, in a specific _Test Case_ we must detail the initial EGO and target speeds, the initial distance between the two cars, the lateral shift, when the target will start to brake, the target deceleration value, if it's an impact mitigation or an impact avoidance case, and so on.
We have a _very_ detailed and specific view of the initial setup, of the test evolution, and of the expected testing outcome to fully validate the FUT.

[.text-center]
130
image::from_test_scenario_to_test_cases.jpg[Test Scenario vs Test Case, 600]
131 132 133 134 135 136 137 138 139 140 141 142
.Test Scenario vs Test Case

.Test Scenario vs Test Case
|===
|*Test Scenario*|*Test case*
|A one, liner possibly associated with multiple _Test Cases_|A name, some pre-conditions, test steps, expected results and post-conditions
|Guides a user on _What to test_|Guides a user on _How to test_
|Tests the end-to-end functionality of a software application|Validates a _Test Scenario_ by executing a set of steps
|Is derived from a _Use Case_|Is derived from a _Test Scenario_
|Consists of high-level actions|Consists of low-level actions
|Easy to maintain, due to the hight-level design|Hard to maintain, due to the heavy specialization
|Less time consumption compared to _Test Cases_|More time consumption compared to _Test Scenarios_
143
|===