Perform a Gremlin Query that asks Syndeia for all known and online Repositories which hold any kind of Requirement and then retrieve the current count of artifacts within the projects of that repository and rank order the resulting set

GremlinGroovyRequirementsReposSizesQuery.groovy

g.V().has( 'ArtifactType', 'sLabel', 'ArtifactType' )
.match( 
  __.as( 'at' ).has( 'name' ).filter{ it.get().value('name') ==~ /(?i).*?requirement.*/ },
  __.as( 'at' ).out( 'ownedBy' ).as( 'or' ), 
  __.as( 'or' ).in( 'ownedBy' ).hasLabel( 'Container' ).as( 'c' ), 
  __.as( 'or' ).has( 'externalId' ).filter{ 
    it.get().value( 'externalId' ).contains( 'http' ) }.filter{ 
    { u -> try { headRequest = new URL( u ).openConnection(); headRequest.setConnectTimeout( 1000 ); headRequest.setRequestMethod( "HEAD" ); return headRequest.getResponseCode() in [ 200, 401 ] } catch (e) { return false } }( it.get().value( 'externalId' ) ) }, 
  __.as( 'or' ).out( 'hasType' ).as( 'rt' ) 
)
.project( 'repo', 'sKey', 'container', 'cKey', 'external' )
  .by( select( keys ).select( 'or' ).values( 'name' ) )
  .by( select( keys ).select( 'or' ).values( 'sKey' ) )
  .by( select( keys ).select( 'c' ).values( 'name' ) )
  .by( select( keys ).select( 'c' ).values( 'externalKey' ) )
  .by( select( keys ).select( 'rt' ).map{ it.get().value( 'name' ).replaceAll( ' Repository Type', '' ).toLowerCase().replaceAll( 'doors-ng', 'doors' ) }.map{ it -> '/external/'.concat( it.get() ) } )
.dedup().by( 'container' )
.project( 'name', 'artifacts' )
  .by( select( values ).map{ it -> "${it.get()[2]} (${it.get()[0]})" } )
  .by( select( values ).map{ it -> { u,eu,et -> try { 
      artifactsGet = new URL( "${syndeiaUrl}${u}" ).openConnection();
      artifactsGet.setConnectTimeout( 1000 ); 
      artifactsGet.setRequestProperty( "Accept", "application/json" ); 
      artifactsGet.setRequestProperty( "Ext-Auth-Token", et ); 
      artifactsGet.setRequestProperty( "User-Id", eu ); 
      artifactsGet.getResponseCode(); 
      j = new groovy.json.JsonSlurper().parseText( artifactsGet.getInputStream().getText() ); 
      return Math.max( j.pageInfo.countOnPage, j.pageInfo.totalCount ) 
    } 
    catch (e) {
      return 0 
    } 
  }( "${it.get()[4]}/${it.get()[1]}/artifacts?container.externalKey=${it.get()[3]}&page=1&pageSize=1",
    *{ rk -> try { 
      authGet = new URL( "{vaultUrl}/${rk}" ).openConnection();
      authGet.setConnectTimeout( 1000 ); 
      authGet.setRequestProperty( "X-Vault-Token", "${vaultToken}" ); 
      authGet.setRequestProperty( "X-Vault-Request", "true" );
      authGet.setRequestProperty( "X-Vault-Namespace", "admin/syndeia/" );
      authGet.getResponseCode(); 
      j = new groovy.json.JsonSlurper().parseText( authGet.getInputStream().getText() ); 
      return [ j.data.data.user, j.data.data.token ]
    } 
    catch (e) {
      return [ "unknown user", "lacking password" ]
    }
  }( it.get()[1] ) ) } )
.order().by( 'artifacts', Order.decr )

Version_without_G-strings.groovy

g.V().has( 'Repository', 'sLabel', 'Repository' )
  .has( 'externalId' )
  .filter { it.get().value( 'externalKey' ) ==~ /.*Syndeia.*RESTful.*/ }
  .values( 'externalId' ).as( 'syndeiaUrl' )
.V().has( 'ArtifactType', 'sLabel', 'ArtifactType' )
.match( 
  __.as( 'at' ).has( 'name' ).filter{ it.get().value('name') ==~ /(?i).*?requirement.*/ },
  __.as( 'at' ).out( 'ownedBy' ).as( 'or' ), 
  __.as( 'or' ).in( 'ownedBy' ).hasLabel( 'Container' ).as( 'c' ), 
  __.as( 'or' ).has( 'externalId' ).filter{ 
    it.get().value( 'externalId' ).contains( 'http' ) }.filter{ 
      { u -> try { headRequest = new URL( u ).openConnection(); 
                   headRequest.setConnectTimeout( 1000 );
                   headRequest.setRequestMethod( "HEAD" );
                   return headRequest.getResponseCode() in [ 200, 401 ]
                 } 
                 catch (e)
                 { 
                   return false 
                 }
      }( it.get().value( 'externalId' ) )
    }, 
  __.as( 'or' ).out( 'hasType' ).as( 'rt' ) 
)
.project( 'repo', 'sKey', 'container', 'cKey', 'external', 'sUrl', 'vUrl', 'vToken', 'vUser' )
  .by( select( keys ).select( 'or' ).values( 'name' ) )
  .by( select( keys ).select( 'or' ).values( 'sKey' ) )
  .by( select( keys ).select( 'c' ).values( 'name' ) )
  .by( select( keys ).select( 'c' ).values( 'externalKey' ) )
  .by( select( keys ).select( 'rt' ).map{ it.get().value( 'name' ).replaceAll( ' Repository Type', '' ).toLowerCase().replaceAll( 'doors-ng', 'doors' ) }.map{ it -> '/external/'.concat( it.get() ) } )
  .by( select( 'syndeiaUrl' ) )
  .by( constant( "https://vault-.hashicorp.cloud:8200" ) )
  .by( constant( "hvs.CAESIDqRxhzexGPxenIjtD NO SOUP FOR YOU XRRZkNwYm0yTjUuMkJkU3UQmLQG" ) )
  .by( constant( "some.user" ) )
.dedup().by( 'container' )
.project( 'name', 'artifacts' )
  .by( select( values ).map{ it -> it.get()[2] + " (" + it.get()[0] + ")" } )
  .by( select( values ).map{ it -> { u,eu,et -> try { 
      artifactsGet = new URL( it.get()[5] + u ).openConnection();
      artifactsGet.setConnectTimeout( 1000 ); 
      artifactsGet.setRequestProperty( "Accept", "application/json" ); 
      artifactsGet.setRequestProperty( "Ext-Auth-Token", et ); 
      artifactsGet.setRequestProperty( "User-Id", eu ); 
      artifactsGet.getResponseCode(); 
      j = new groovy.json.JsonSlurper().parseText( artifactsGet.getInputStream().getText() ); 
      return Math.max( j.pageInfo.countOnPage, j.pageInfo.totalCount ) 
    } 
    catch (e) {
      return 0 
    } 
  }( it.get()[4] + "/" + it.get()[1] + "/artifacts?container.externalKey=" + it.get()[3] + "&page=1&pageSize=1",
    *{ rk -> try { 
      authGet = new URL( it.get()[6] + "/v1/kv/data/" + it.get()[8] + "/" + rk ).openConnection();
      authGet.setConnectTimeout( 1000 ); 
      authGet.setRequestProperty( "X-Vault-Token", it.get()[7] ); 
      authGet.setRequestProperty( "X-Vault-Request", "true" );
      authGet.setRequestProperty( "X-Vault-Namespace", "admin/syndeia/" );
      authGet.getResponseCode(); 
      j = new groovy.json.JsonSlurper().parseText( authGet.getInputStream().getText() ); 
      return [ j.data.data.user, j.data.data.token ]
    } 
    catch (e) {
      return [ "unknown user", "lacking password" ]
    }
  }( it.get()[1] ) ) } )
.order().by( 'artifacts', Order.decr )

==>{name=Syndeia Test (Jama @ Intercax), artifacts=5336}
==>{name=Syndeia Test CLONE (Jama @ Intercax), artifacts=2876}
==>{name=Syndeia Applications (Jama @ Intercax), artifacts=184}
==>{name=Automobile (Jama @ Intercax), artifacts=91}
==>{name=Spacecraft (Jama @ Intercax), artifacts=29}
==>{name=Unmanned Aerial Vehicle (Jama @ Intercax), artifacts=26}
==>{name=JasClm702_CmEnabled_RmProject (DNG 7.0.2 @ Intercax CM-enabled (JAS, JTS, CCM, "RM", QM, LQE, GC("CM"))), artifacts=1}
==>{name=Sample Lifecycle Project (Requirements) (DNG 6.0.6 @ Intercax CM-enabled ("super-clm606") (JTS, CCM, "RM", QM, LQE, GC("CM"))), artifacts=0}
==>{name=Non-CM Project for Demo 2021 may 19 (DNG 6.0.6 @ Intercax CM-enabled ("super-clm606") (JTS, CCM, "RM", QM, LQE, GC("CM"))), artifacts=0}
==>{name=CM-Enabled Requirements Project (DNG 6.0.6 @ Intercax CM-enabled ("super-clm606") (JTS, CCM, "RM", QM, LQE, GC("CM"))), artifacts=0}
==>{name=Non-CM Lifecycle (Requirements) (DNG 6.0.6 @ Intercax CM-enabled ("super-clm606") (JTS, CCM, "RM", QM, LQE, GC("CM"))), artifacts=0}
==>{name=Carys - Unmanned Aerial Vehicle (Jama @ Intercax), artifacts=0}
==>{name=Big Data Medical Device (Jama @ Intercax), artifacts=0}
==>{name=Integrated System (Jama @ Intercax), artifacts=0}
==>{name=DriveLine (Jama @ Intercax), artifacts=0}
==>{name=Syndeia Test Automation (RO) (Jama @ Intercax), artifacts=0}
==>{name=Medical Device (Jama @ Intercax), artifacts=0}
==>{name=Test Project for IHD-2471 (DNG 7.0.2 @ Intercax CM-enabled (JAS, JTS, CCM, "RM", QM, LQE, GC("CM"))), artifacts=0}
==>{name=JasClm702_NonCM_RequirementsProject (DNG 7.0.2 @ Intercax CM-enabled (JAS, JTS, CCM, "RM", QM, LQE, GC("CM"))), artifacts=0}
==>{name=Sandbox for Intercax Team (DNG 7.0.2 @ Intercax CM-enabled (JAS, JTS, CCM, "RM", QM, LQE, GC("CM"))), artifacts=0}
==>{name=Intercax Project (Jama @ Partner), artifacts=0}
==>{name=Syndeia Test Project (JIRA @ Partner S), artifacts=0}
==>{name=Cybersecurity (JIRA @ Partner S), artifacts=0}
==>{name=Configuration Management Database (JIRA @ Partner S), artifacts=0}
==>{name=Requirements (JIRA @ Partner S), artifacts=0}
==>{name=Milestones (JIRA @ Partner S), artifacts=0}

This is a multi-phase query.

It begins by entering the Syndeia Graph looking for ArtifactType vertices.

From these, it filters down to only those that have some capitalization of the word "requirement" in their type name.

For these "requirement" ArtifactTypes, it finds the Repository instances that own those types.

From the Repositories, it finds the type of each such Repository.

For these Repository Types, it gets the name fragment that provides the path segment for the Syndeia /external/:repoType endpoint.

For each Repository, it finds its set of Containers - these are the individual projects or workspaces within the repositories.

It filters the set of Containers by removing those that are in Containers whose Repository isn't online accessible at the current moment.

Having found a set of Containers in live Repositories which hold kinds of Requirements, it projects out the set of repository, repository key, container, container key, and the URL path for the external access through Syndeia to the repository that holds the container.

It removes duplicates from this set by discriminating on the Container sKeys.

It then begins to collapse the 5-property sets to 2-property ones to yield a set of name, artifacts pairs where the name is the name of the Container and its Repository and the artifacts is the total count of resources within the particular Container and Repository.

Forming the name is trivial: it simply concatenates the name of the Container with the name of the Repository wrapped in parentheses.

To obtain the count of artifacts within the particular Container, it constructs a REST query to Syndeia for the /external/:repoKind/artifacts endpoint where the Container is found and includes the path parameter that provides the Container sKey.

The response of the REST query is parsed as JSON and the totalCount, if any is returned.

To make the REST call through Syndeia to the external repositories, Syndeia needs the user's specific credentials for each repository. Those credentials are stored in a REST-accessible Hashicorp Vault.

Therefore, a separate call is first made to Hashicorp Vault to retrieve the user credentials for the specific repository for the current user. Syndeia doesn't store the user credentials, it only obtains them dynamically, in the midst of the query, when needed, securely.

The query concludes by ordering the set by most to least artifacts.

Any repository which is not accessible or doesn't offer a totalCount is reported as having no (0) artifacts.