A specification for declaring vulnerabilities in intentionally vulnerable applications ("vulnapps") to help scanners assess their own find rate

.vulns.yml.md

.vulns.yml

A specification for declaring vulnerabilities in intentionally vulnerable applications ("vulnapps") to help scanners assess their own find rate.

Usage Concept

1. A vulnerable application declares its vulnerabilities in an agreed schema in a .vulns.yml (or .vulns.json) file
2. A security scanner analyzes the vulnapp and generates a report with its findings
3. The individual findings in the report are checked for flags specified in .vulns.yml to assess them as false negative/positive or true positive

Schema Drafts

1. .vulns.flags-only.yml: Flags only, no further grouping or categorization
2. .vulns.subtyped-flags.yml: Flags categorized into different types
3. .vulns.grouped-flags.yml: Flags grouped by their location of occurence with optional extra attributes (like HTTP method)

Draft Comparison

Draft	Declaration EEffort	Flag Duplication	Report Matching Effort	Assessment Precision	Extensibility
Flags only	Low (:heavy_plus_sign:)	Low (:heavy_plus_sign:)	Low (:heavy_plus_sign:)	Low (:heavy_minus_sign:)	Low (:heavy_minus_sign:)
Subtyped flags	Medium	Medium	High (:heavy_minus_sign:)	Medium	Medium
Grouped flags	High (:heavy_minus_sign:)	High (:heavy_minus_sign:)	Medium	High (:heavy_plus_sign:)	High (:heavy_plus_sign:)

Discussion

Most of the discussion that led to the creation of these three drafts happened in juice-shop/juice-shop#1441 and on https://owasp.slack.com/archives/CPMEWT342/p1597085979033700.

vulns.flags-only.yml

application:
  name: OWASP Juice Shop
  source: https://github.com/bkimminich/juice-shop
vulnerabilities:
  -
    description: DOM-based XSS in search query
    classifiers:
      - "CWE-79"
      - "CWE-749"
      - "A7:2017"
      - "WASC-8"
    flags:
      - "main-es[0-9]*\.js"
      - "search-result\.component\.html"
      - "search-result\.component\.ts"
      - "\[innerhtml\]"
      - "DomSanitizer"
      - "searchValue"
      - "bypassSecurityTrustHtml"
      - "#/search"
      - "q="
  -
    description: Stored XSS in product description
    classifiers:
      - "CWE-79"
      - "CWE-749"
      - "A7:2017"
      - "WASC-8"
    flags:
      - "main-es[0-9]*\.js"
      - "search-result\.component\.ts"
      - "product-details\.component\.html"
      - "DomSanitizer"
      - "tableData\[i\]\.description"
      - "bypassSecurityTrustHtml"
      - "trustProductDescription"
      - "\[innerhtml\]"
      - "/api/Products"
      - "/api/Products/[0-9]*"
      - "#/search"
  -
    description: SQL Injection into user login
    classifiers:
      - "CWE-74"
      - "CWE-89"
      - "A1:2017"
      - "WASC-19"
    flags:
      - "login\.js"
      - "sequelize\.query"
      - "req\.body\.email"
      - "req\.body\.password"
      - "/rest/user/login"
      - "{email: \".*\", password: \".*\"}"
  -
    description: XXE via file upload in user complaints
    classifiers:
      - "CWE-611"
      - "A4:2017"
      - "WASC-43"
    flags:
      - "fileUpload\.js"
      - "handleXmlUpload"
      - "libxml\.parseXml"
      - "req\.body\.password"
      - "B2B customer complaints via file upload have been deprecated for security reasons"
      - "/file-upload"
      - "Content-Disposition: form-data; name=\"file\";"
      - "Content-Type: text/xml"
  -
    description: Stored XSS in customer feedback through non-recursive HTML sanitization
    classifiers:
      - "CWE-79"
      - "CWE-829"
      - "A7:2017"
      - "A9:2017"
      - "WASC-8"
    flags:
      - "package\.json"
      - "main-es[0-9]*\.js"
      - "contact\.component\.ts"
      - "contact\.component\.html"
      - "administration\.component\.html"
      - "administration\.component\.ts"
      - "feedback-details\.component\.html"
      - "about\.component\.html"
      - "about\.component\.ts"
      - "DomSanitizer"
      - "feedbacks\[i\]\.comment"
      - "feedback\.comment"
      - "bypassSecurityTrustHtml"
      - "populateSlideshowFromFeedbacks"
      - "\[innerhtml\]"
      - "sanitize-html.*1\.4\.2"
      - "/api/Feedbacks"
      - "comment: \".*\""
      - "/#/administration"
      - "/#/about"

vulns.grouped-flags.yml

application:
  name: OWASP Juice Shop
  source: https://github.com/bkimminich/juice-shop
vulnerabilities:
  -
    description: DOM-based XSS in search query
    classifiers:
      - "CWE-79"
      - "CWE-749"
      - "A7:2017"
      - "WASC-8"
    evidences:
      -
        location: "#/search"
        flags:
          - "q="
      -
        location: "main-es[0-9]*\.js"
        flags:
          - "bypassSecurityTrustHtml"
      -
        location: "search-result\.component\.ts"
        flags:
          - "DomSanitizer"
          - "searchValue"
          - "bypassSecurityTrustHtml"
      -
        location: "search-result\.component\.html"
        flags:
          - "\[innerhtml\]"
          - "searchValue"
  # [...]
  -
    description: SQL Injection into user login
    classifiers:
      - "CWE-74"
      - "CWE-89"
      - "A1:2017"
      - "WASC-19"
    evidences:
      -
        location: "login\.js"
        flags:
          - "sequelize\.query"
          - "req\.body\.email"
          - "req\.body\.password"
      -
        location: "/rest/user/login"
        http_method: POST
        flags:
          - "{email: \".*\", password: \".*\"}"
  # [...]

vulns.subtyped-flags.yml

application:
  name: OWASP Juice Shop
  source: https://github.com/bkimminich/juice-shop
vulnerabilities:
  -
    description: DOM-based XSS in search query
    classifiers:
      - "CWE-79"
      - "CWE-749"
      - "A7:2017"
      - "WASC-8"
    flags:
      files:
        - "main-es[0-9]*\.js"
        - "search-result\.component\.html"
        - "search-result\.component\.ts"
      code:
        - "\[innerhtml\]"
        - "DomSanitizer"
        - "searchValue"
        - "bypassSecurityTrustHtml"
      runtime:
        - "#/search"
        - "q="
  -
    description: Stored XSS in product description
    classifiers:
      - "CWE-79"
      - "CWE-749"
      - "A7:2017"
      - "WASC-8"
    flags:
      files:
        - "main-es[0-9]*\.js"
        - "search-result\.component\.ts"
        - "product-details\.component\.html"
      code:
        - "DomSanitizer"
        - "tableData\[i\]\.description"
        - "bypassSecurityTrustHtml"
        - "trustProductDescription"
        - "\[innerhtml\]"
      runtime:
        - "/api/Products"
        - "/api/Products/[0-9]*"
        - "#/search"
  -
    description: SQL Injection into user login
    classifiers:
      - "CWE-74"
      - "CWE-89"
      - "A1:2017"
      - "WASC-19"
    flags:
      files:
        - "login\.js"
      code:
        - "sequelize\.query"
        - "req\.body\.email"
        - "req\.body\.password"
      runtime:
        - "/rest/user/login"
        - "{email: \".*\", password: \".*\"}"
  -
    description: XXE via file upload in user complaints
    classifiers:
      - "CWE-611"
      - "A4:2017"
      - "WASC-43"
    flags:
      files:
        - "fileUpload\.js"
      code:
        - "handleXmlUpload"
        - "libxml\.parseXml"
        - "req\.body\.password"
        - "B2B customer complaints via file upload have been deprecated for security reasons"
      runtime:
        - "/file-upload"
        - "Content-Disposition: form-data; name=\"file\";"
        - "Content-Type: text/xml"
  -
    description: Stored XSS in customer feedback through non-recursive HTML sanitization
    classifiers:
      - "CWE-79"
      - "CWE-829"
      - "A7:2017"
      - "A9:2017"
      - "WASC-8"
    flags:
      files:
        - "package\.json"
        - "main-es[0-9]*\.js"
        - "contact\.component\.ts"
        - "contact\.component\.html"
        - "administration\.component\.html"
        - "administration\.component\.ts"
        - "feedback-details\.component\.html"
        - "about\.component\.html"
        - "about\.component\.ts"
      code:
        - "DomSanitizer"
        - "feedbacks\[i\]\.comment"
        - "feedback\.comment"
        - "bypassSecurityTrustHtml"
        - "populateSlideshowFromFeedbacks"
        - "\[innerhtml\]"
        - "sanitize-html.*1\.4\.2"
      runtime:
        - "/api/Feedbacks"
        - "comment: \".*\""
        - "/#/administration"
        - "/#/about"

Please provide feedback here via comments and/or by 👍/👎 reactions to this comment. Thank you!

Hi @bkimminich,

Is there any progress on this further ?

thanks,
Karan