Skip to content

Instantly share code, notes, and snippets.

@MatteoRagni
Last active February 16, 2024 13:20
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save MatteoRagni/cfaffe48e82ba690fa8b2ec817811358 to your computer and use it in GitHub Desktop.
Save MatteoRagni/cfaffe48e82ba690fa8b2ec817811358 to your computer and use it in GitHub Desktop.
ASAM Notes

Proposal for rules UID

The rule UID can be defined in a unique form, so it can be identified and optionally encapsulate the identity and the revision of the rule that is taken into consideration.

Concepts

In the design of such UID we must take into consideration the following:

  • even if the majority of rules are defined by ASAM, there is nothing that prevents third-party to define their own rules (concept of emanating entity)
  • rules can apply to FRAMEWORK (e.g., existence of a file, extension of the file, etc.), OpenDRIVE and OpenSCENARIO. There is nothing that prevents the creation of rules for other standards such as, e.g., OTX. The UID should allow future extensions (concept of reference standard)
  • ⁠a rule can be mandated directly from the standard, or may be implemented in the checker, to be proposed in the standard. Furthermore a rule can apply only to certain version of the standard (concept of definition setting)
  • a rule can optionally be part of a certain group or subgroup of rules that refer to the same baseline concepts (concept of rule-set)
  • rules must be addressable inside their definition setting or rule-set (concept of name)
  • rules can evolve in time (for example in description) but in general the underline concept of a rule should always hold. Nevertheless it is advisable to maintain a version for the rule. It should be debated if the version should be included or not, but the proposal tries to be flexible enough to include this information (concept of version)

In general, rules will be queried: querying should allow to move from generic to specific information. A query that can be performed directly on UID may be advisable.

The proposal considers a notation in wich concept are separated by a well defined character, like :

Example

Lets start with an example for a very complete UID, and lets break down its content. For the reference example a rule listed in the subsection Rules of one of the ASAM Standards for OpenDRIVE is used. This subsections were introduced for the first in the revision 1.6 of the standard.

ASAM OpenDRIVE Standard 1.6.0, Chapter 7.2 - Road Reference Line, subsection Rule, first point:

Each road shall have a reference line

Starting from the initial concept of emanating entity, it is possible to use this notation to actually give an UID for the quoted rule:

  • emanating entity: ASAM net.asam
  • reference standard: OpenDRIVE xodr
  • definition settings: as a proposal to be discussed, one can consider to use the version of the standard in which the rule firstly appeared, in this case 1.6.0
  • ruleset: Road Planview Geometry definitions road.planview.geometry
  • name: a name. It can be arbitrary chosen, e.g.: rl-exists for a more descriptive name, or r-001 for something more standardisable.
  • version: it is advisable to use something simple, like a number

The complete UID would be something like:

asam.net:xodr:1.6.0:road.planview.geometry.rl-exists:1
-------- ---- ----- ---------------------- --------- -
 |        |    |     |                      |        |
 entitity | version  |                    name     rule-version (optional, simplified)
       standard    rule-set (name is always last in rulest dot-notation)

Regarding the validation schema itself, only the the first concept is fully required. Entities that are not ASAM itself are not required to declare rule with the concepts stated above. The schema is a formalism which is required only for rule under asam.net:*. To be noticed that the proposed schema suggests implicitly a hierarchy of concepts, from the broadest one (entity) to the most specific one (rule version).

Example: if my company has defined it's own rules, they may be something like: com.antemotion.rule-1 and com.antemotion.rule-2 and the UID should be considered valid.

Example: the workgroup ASAM for OpenDRIVE shall create all UIDs for the rules under the reverse domain name that starts with net.asam.xodr.

ASAM Concepts

  • entity: asam.net
  • standard:
    • Framework: asam.net:qc
    • OpenDRIVE: asam.net:xodr
      • v1.4.0: asam.net:xodr:1.4.0
      • v1.5.0: asam.net:xodr:1.5.0
      • v1.6.0: asam.net:xodr:1.6.0
      • v1.6.1: asam.net:xodr:1.6.1
      • v1.7.0: asam.net:xodr:1.7.0
      • v1.8.0: asam.net:xodr:1.8.0
      • Future v2?: asam.net:xodr:2.0.0
      • Rules that are part only of Quality Checker and are proposed for insertion in standard: asam.net:xodr:future
    • OpenSCENARIO XML: asam.net:xosc
      • v1.0.0: asam.net:xosc:1.0.0
      • v1.1.0: asam.net:xosc:1.1.0
      • v1.1.1: asam.net:xosc:1.1.1
      • v1.2.0: asam.net:xosc:1.2.0
    • OpenSCENARIO DSL?: asam.net:osc
      • v2.0.0?: asam.net:osc:2.0.0

All the other sub-concepts for ASAM are subject to discussion during actual definition of rules.

Query rules

The proposed formalism allow to perform query on rules (or set of rules) using UNIX style wildcards notation (python fnmatch). In the implementation of the checker framework is advisable that the end-user can enable or disable check based on the UID of the rules that the check verifies. This approach pushes agnostic approaches for check selection (i.e., for checker framework different with respect to the one offered by the ASAM OS project).

Examples:

  • select all the OpenDRIVE geometrical rules that are not in version 2 of the standard: asam.net:xodr:1.?.0:*geometry.* matches asam.net:xodr:1.6.0:road.planview.geometry.rl-exists:1
  • select all the OpenDRIVE geometrical rules that are in version 1.6 and 1.7 of the standard: asam.net:xodr:1.[67].0:*geometry.* matches asam.not:xodr:1.6.0:road.planview.geometry.rl-exists:1 but does not match asam.net:xodr:1.8.0:road.planview.geometry.rl-exists:1

OpenDRIVE QC Sub group

The main task inside the OpenDRIVE QC subgroup is to define the best practice for naming OpenDRIVE rules, taking for granted the root domains asam.net:xodr:?.?.?. The concepts that should be defined are:

  • rule-sets: there are different alternatives for defining the rule-set:
    • OpenDRIVE uses XML as main language, and the majority of the rule may express a hierarchy that strongly resemble the hierarchy of the XML schema of opendrive (i.e., xml elements t_road > t_road_type > t_road_type_speed may be defined as road.type.speed_type subdomain). Other special cases should be discussed.
    • Chapter inside standard description may be used (discouraged by members of the workgroup)
    • Special definitions may be defined by the subgroups for each rule
  • rule naming convention: in general is advisable to use a short string name that describes what the rule is doing. Alternatively, the rule can be numbered based upon definition order. As a final alternative, rule name can be generated as short sha256 (first 7 characters of sha256) of the description field.

Weaknesses and Pain Point

  1. The UID is very long. It can be a problem to search for it efficiently in Databases (it is not an integral type nor has a constant size) or may raise Path Too Long error (I'm looking at you Windows).

    The length of the UID comes with its advantages. It is possible to understand many of the information of a rule only by looking at its uid, even without reading the rule description. Furthermore, UID can be navigated much like a file system. It is possible to change base location using both an absolute or a relative reference (relative reference may start with a . character), or list all the sub-uid from that location.

    The total number of rules is low enough that using this string in query for databases is a non-issue in terms of efficiency. In case specific implementations needs a faster query strategy, a checksum of the original UID can be used as primary key (integral type with constant size).

    Using a directory structure that mimics the structure of the uid, or using the entire uid for filenames can reach rapidly numbers that triggers the Path too long error in windows. This must be taken in consideration in all implementations (even in documenting the project)

  2. The version of the standard in the UID may create confusion

    This is the biggest weak point in the proposal. There are two main confusing situation:

    • from an end user perspective, request/remove the check for a certain rule requires a knowledge of the first version of the standard in which the rule appeared. Must be taken into consideration that to actually validate a rule, its uid is not sufficient. Indside the definition of the rule there will be a range of version on which the rule can be validated, and the current file should be inside that range.
    • from a developer perspective may be difficult to undestand which rule id insert in a checker.

    Alongside this confusion there is a great advantage: if something changes drammatically between two standards, there can be two different rule id (that have the same rule-set and rule name) relative to two different rules. The different part is the version of the standard in the UID (asam.net:xoder:1.6.0:road.planview.geometry.rl-exists:1 != asam.net:xoder:1.8.0:road.planview.geometry.rl-exists:1). The probability that this will happen is quite low.

  3. Is a simple counter for versions enough, or do we need a semantic versioning?

    No. We can also use a simple counter in place of semantic versioning. Please notice that the inclusion of the version of the rule in the uid is still open for discussion

ASAM Framework Homework

ESMINI Viewer Implementation

This is a very draft analysis performed in around 10 minutes on the code of ESMINI, on the master` branch of the codebasehosted on github. Please notice that I have done no dependencies analysis and no LICENSE analysis

It appears that ESMINI is using OpenSceneGraph for visualization of the OpenDRIVE files. AFAIK this module is compatible with all the major platform, and has a limited support also for ARM (mainly Android and iOS) architecture but there is no specific WebAssembly / EMScripte implementation that means no browser support. The base code for the viewer is included in the directory Modules/VieverBase and includes 3 translation units:

  • RubberBandManipulator: implementation of the camera controls. The name may refer to the implementation of a camera that follows an entity (like a vehicle during a scenario simulation), and the camera arm that connects arm and entity is not rigid, but has a dynamic that filters out vibrations.

  • viewer: appeaers as the main entrypoint for the viewer, and contains the implementation for various entities shown in the viewer, alongside some utilities for the viewer itself. The main object is viewer::iewer. The viewer is built with the following:

    • an opendrive roadmanager::OpenDrive *, see next point, and for your enjoiment this is a pointer, and the pointer is copied in a private member variable... yay!
    • a string that contains a name for the environment model (which I should investigate a little further),
    • a string that contains the name for the scenario (which appears to be unused),
    • the executable path (which is used for loading resources, by using a relative path... maybe it is used to understand the installation path of the whole application programmatically?)
    • an osg::ArgumentsParser, which is responsible to parse options specific for OpenSceneGraph command line arguments, which appears to be a subset of the osgviewer options: --window (window size dimensions), --borderless-window (size for a borderless window), --screen (sets the screen to to use, when multiple screen are available), --lodScale, --singleThreaded, --CullDrawThreadPerContext, --DrawThreadPerContext, --CullThreadPerCameraDrawThreadPerContext, --clear-color.
  • roadgeom: it appears the main handler for drawing the geometry of the opendrive. It defines an object called RoadGeom with no default ctor, requires a roadmanager::OpenDrive * object as input (a bare pointer... they like to live dangerously, I suppose). At construction, textures for asphalt and grass are created from file, and statts to create the lane for allgeometries, iterating over Roads / LaneSections / Lanes. The geometrical construction (points) are obtained directly from the roadmanager::OpenDrive object (on lanes, method GetOSIPoints, returns a OsiPoints* - glom - which is a thin layer on a std::vector<roadmanager::PointStruct> object). roadmanager::PointStruct contains bare values for s, x, y, z, h (they are undocumented but we can assume what they are). All this long analysis makes me saying the following:

    the ESMINI viewer is strictly coupled with the ESMINI parser for the OpenDRIVE

ODRViewer Implementation

This wants to remain private.

ODRViewer is implemented by crosscompiling the main dependency libOpenDRIVE using WASM / Emscripten. The visualization is done in the page in a WebGL canvas. I think the author removed the majority of the code from the main repository, and there are several building blocks that are missing (the only available branch is the public one which is missing some of the necessary code to run a functioning viewer). Still it is possible to find a single forks with a version which appears to be containing partially the code of the online version (still limited). In order to protect this codebase, I made a fork on my personal account.

There are two important missing set of information in the original codebase:

  • the file viewer.js, which is responsible to sideload the WASM module viewer.wasm which at least contains the compiled libOpenDRIVE code. The file implements all the camera controls and loops to actually fill the canvas. The code must be demangled to actually see its behavior, and it is quite complex. From the demangling we can understand that from the compiled module the following are loaded:
    • the actual viewer code is compiled in the viewer.wasm. It seems like the viewer is using some sort of GLFW interface for actual visualization
    • the code for libOpenDRIVE
  • the file viewer.data contains a binary version of the default OpenDRIVE shown in the page when loading in a browser
  • the file viewer.wasm: where the magic happens. I tried to decompile and object-dump the content of the file but th file was actually stripped from known symbol. No reverse engineering with a lot effort.

Let's make a run on the other code linked in the repo.

The main library is libOpenDRIVE and (according to its readme) provides OpenDRIVE parsing and 3D model generation. The model is the used in the viewer. To do so has two main dependencies:

  • pugixml (an xml parser) which is MIT licensed
  • earcut (triangulation library for meshes generation) which is ISC Licensed by Mapbox. It appears to be functionally equivalent to MIT License

YES... BUT... no Viewer code in here. So let's go back in time for older versions, starting with releases. And here the magic of release 0.3 happens: the code contains the visualization code, even if I'm quite sure it is not in par with the one of the current release.

Older version used three.js library as dependencies to show th content and to control the camera in orbit mode. Additionally, the GUI controls were implemented using DAT.gui, which is very similar to dear-imgui (which I don't know why but I strongly suspect is the widget implementation in the current version online - EDIT: with string viewer.asm I found the initialization commands od DearImGUI, in particular: /Users/sebastianpagel/dev/projects/odrviewer/thirdparty/imgui/imgui_impl_glfw.cpp file is included in wasm). The viewer of the current version seems to implement objects in a C++ namespace odr::viewer. This is a current list of possible exported object implemented by the author:

  • odr::viewer::Object3D
  • odr::viewer::Mesh
  • odr::viewer::Line
  • odr::viewer::ViewerObject
  • odr::viewer::TriggerableObject
  • odr::viewer::ViewerLaneSection
  • odr::viewer::DrawableObject
  • odr::viewer::PickableObject << WHAT?? Pickable ala python??
  • odr::viewer::XmlObject
  • odr::viewer::TreeViewActions
  • odr::viewer::ViewerLane
  • odr::viewer::ViewerRoadNetwork
  • odr::viewer::Viewer
  • odr::viewer::LaneSectionViewerObject
  • odr::viewer::LaneViewerObject
  • odr::viewer::RoadMarkViewerObject
  • odr::viewer::RoadObjectViewerObject
  • odr::viewer::RoadSignalViewerObject
  • odr::viewer::RoadViewerObject
  • odr::viewer::HidableObject
  • odr::viewer::ViewerRoadmark
  • odr::viewer::ViewerRoadObject
  • odr::viewer::ViewerRoadSignal
  • odr::viewer::ViewerRefLine
  • odr::viewer::ViewerRoad

Like ESMINI viewer, odrviewer is strictly coupled with the libOpenDRIVE library. In this case, the viewer is completely hidden and it is not made available.

Nuitka

Converts python code to executable objects (DLL, EXE or SO, ELF with main). May create compiled executable

https://nuitka.net/doc/user-manual.html#use-cases

@JonasConrad-PES
Copy link

JonasConrad-PES commented Feb 5, 2024

asam.net:xodr:1.6.0:road.planview.geometry.rl-exists:1
-------- ---- ----- ---------------------- --------- -
 |        |    |     |                      |        |
 entitity | version  |                    name     rule-version (optional, simplified)
       standard    rule-set (name is always last in rulest dot-notation)

Thank you very much for your proposal.
I personally like the structure of the second version better.
Also, the simplified rule version helps with making it more distinct from the xodr version.

I have one suggestion in reagards to the "rule-set".
What do you think of using the chapter headlines of the OpenDrive document instead of the structure of the the xml file?

i.e. instead of:

XY:road.planview.geometry.rl-exist:1

it would look a little something like this:

XY:Geometry.RoadRefLn.rrl-exist:1

This might make it more easy to find the rule definition in the standard.
But if future versions of the standard have a completely different chapter structure / different headlines it could cause problems.

@MatteoRagni
Copy link
Author

MatteoRagni commented Feb 7, 2024

I agree with you on sticking on the second version, and I think it is what we should propose.

Regarding the ruleset, I have no preference on the content itself, but I would force format to be compatible with one of the two regexps:

  • ([a-z0-9-]+\.)+ (e.g. geometry.roadrefln.rrl-exists, only lower case, dot separated, kebab case)
  • ([a-z0-9_]+\.)+ (e.g. geometry.roadrefln.rrl_exists, only lower case, dot separated, snake case, python namespace compatible)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment