Doxygen Notes for umlaut AP handling of requirements traceability:

doxygen-req-apporach.txt

Doxygen Notes for umlaut AP handling of requirements traceability:

UmlautSoftwareDevelopmentAccount/AUTOSAR-Adaptive#10


> Arch spec is in doc/arch.doc
  
  Comments can either belong to @brief or do not belong to @brief.

  Doxygen parses laguages constructs to extract comments and store them
  in Entry types (src/entry.h). Entry types are the main source of information,
  including line numbers, file names, groups, etc. Note that Entry object may
  carry sequence of "subentries", thus it may have "children", a hierarchy.
  
    class Entry
    {
      ...
      QCString     doc;         //!< documentation block (partly parsed)
      ...
      QCString     docFile;     //!< file in which the documentation was found
      QCString     brief;       //!< brief description (doc block)
      ...
      QCString     briefFile;   //!< file in which the brief desc. was found
      QCString     inbodyDocs;  //!< documentation inside the body of a function
    ...

  Data stored in Entry types is posprocessed by "gather information" step
  in function parseInput from src/doxygen.h by the set of dedicated functions
  (e.g. buildClassList, buildClassDocList, buildVarList, etc.). Those dedicated
  functions use global static data defined in class Doxygen for storage and flow
  of data and control (i.e. the worst possible way of doing programming).
  
    class Doxygen
    {
      public:
        static ClassSDict               *classSDict;
        static ClassSDict               *hiddenClasses;
        static PageSDict                *exampleSDict;
        static PageSDict                *pageSDict;
        static PageDef                  *mainPage;
        static bool                      insideMainPage;
        static FileNameLinkedMap        *includeNameLinkedMap;
    ...

  Language parser parses the code to produce Entry objects, but do not parse
  the structure of the comments directly (i.e. the Doxygen "special commands",
  like @param, @return, etc.). Those "special commands" are parsed by
  src/commentscan.l invoked by the language parser in src/scanner.l file.
  
  From src/doxygen.cpp where the language parser is invoked:
  
    g_s.begin("Parsing files\n");
    if (Config_getInt(NUM_PROC_THREADS)==1)
    {
      parseFilesSingleThreading(root);
    }
    else
    {
      parseFilesMultiThreading(root);
    }
    g_s.end();
  
  Actual parsing of special commands is done at later stages in src/scanner.l
  file which shall be extended with @req handler.

  To make the development straightforward, I propose to reuse the infrastructure
  for @xrefitem and treat @req as @todo, with that exception that in
  the "gather information" REQ_PROVIDER is called to fetch the details for given
  <req-id>, and that information is used in the output produced by @xrefitem
  generators. See https://www.doxygen.nl/manual/commands.html#cmdxrefitem
  
    \xrefitem <key> "(heading)" "(list title)" { text }
  
  To achieve that I propose (in src/commentscan.l):
    - extend docCmdMap with "req" CommandSpacing::Block,
    - extend XRefKind with XRef_Req,
    - tweak setOutput to handle XRef_Req,
    - write handleReq to parse @req,
    - pass the parsed info to setOutput with prepared OutputContext
      for OutputXRef; this way we will get an index of reqs linked 
      to code for free,

  And in the src/doxygen.cpp:
    - extend "gather information" step for xrefitems to fetch the req
      details by means of REQ_PROVIDER:
    
          g_s.begin("Adding xrefitems...\n");
          addListReferences();
          generateXRefPages();
          g_s.end();


> Extending configuration:

  config.xml is used to generate configvalues.h
  which is included by src/config.h that
  exposes macros to control configuration: Config_XXX
  
  Extend config.xml to add new configuration options.
  
> Executing an external application

  Doxygen provides Portable::system to execute an external CLI application.
  More functions exist in Portable namespace.

  Example: src/doxygen.cpp inside "generate documentation" step:
  
    if (generateHtml &&
        Config_getBool(GENERATE_HTMLHELP) &&
        !Config_getString(HHC_LOCATION).isEmpty())
    {
      g_s.begin("Running html help compiler...\n");
      QString oldDir = QDir::currentDirPath();
      QDir::setCurrent(Config_getString(HTML_OUTPUT));
      Portable::setShortDir();
      Portable::sysTimerStart();
      if (Portable::system(Config_getString(HHC_LOCATION), "index.hhp", Debug::isFlagSet(Debug::ExtCmd))!=1)
      {
        err("failed to run html help compiler on index.hhp\n");
      }
      Portable::sysTimerStop();
      QDir::setCurrent(oldDir);
      g_s.end();
    }

  Unfortunately, there is not possiblity to fetch the output of the command.
  Either an extra file is required to pass the information, or custom
  abstraction over the pipe to exchange the data.
  
  Another idea is to use curl (integrate, and then just call REST API). Having
  that it would be possible to define
  
    REQ_PROVIDER=http://<address:port>/doxygen/req
    
  and just issue a GET to doxygen/req/<req-id> to get the requirement information.
  
  In that case, user is required to start a tiny server application that will
  accept REST requests to doxygen/req/<req-id> resources at <address:port>.
  Such an application can be written in any language.



-- insooth 2020-09-23