Description of testing concepts / etc. in Syntribos

syntribos-terminology.md

Terminology for Syntribos

Test

• Meaning: A set of checks that is conducted to uncover the presence of a specific type of vulnerability. Tests should be able to specify requests or Request Chains that it knows how to interact with
• Purpose: Identify vulnerabilities
• Example: SQL Injection

Test Case

• Meaning: One atomic piece of a Test that may or may not reveal a specific vulnerability
• Purpose: Identify specific requests that will trigger a potential vulnerability
• Example: Injecting the string "<script>alert(1)</script>" as a POST variable

Checks

• Meaning: A standardized "check" for a single data point, to increase/decrease confidence in a given finding
• Purpose: Use the same mechanisms for sending/inspecting requests/responses to prevent sub-optimal solutions and code duplication
• Example: Compare a fuzzed request's response time to a vanilla request's response time, and see if it is within our range of acceptable variance
• Related Terms:
  • Test modularization

Signal

• Meaning: Indication of a meaningful piece of information. Tests can specify signals that they are interested in, and they assign point values to given signals. These point values determine "issue confidence." A signal is a float between 0 and 1, based on the context and the strength of the signal. Signals have "slugs" and "tags" associated with them.
• Purpose: Use a standardized way of detecting useful pieces of information about requests/responses
• Examples:
  • HTTPStatusCode
  • HTTPFailure
  • LengthDifferential

Slug

• Meaning: A string, in a standard format, that is assigned to any given signal based on context.
• Purpose: Allow tests to specify slugs as proxies for signals.
• Example:
  • HTTP_STATUS_CODE_500
  • HTTP_FAIL_TIMEOUT
  • LENGTH_DIFF_5X

Request Template

• Meaning: A raw HTTP request (in text format) that contains markup that's used by Syntribos to conduct Tests
• Purpose: Provide Syntribos with the data necessary to interact appropriately with an API, and test all relevant injection points
• Example: A GET request that specifies how to retrieve a user from Keystone

Response Template

• Meaning: A raw HTTP response (in text format) that contains markup that's used by Syntribos to determine whether a potential vulnerability has been uncovered
• Purpose: By comparing actual API responses to the Response Template, Syntribos can determine with greater certainty whether a vulnerability has been found
• Example: An HTTP/201 response that includes several JSON keys for a successful user creation request in Keystone

Request Chain / Request Flow (NEED TO PICK ONE NAME)

• Meaning: A set of Request Templates that represent a workflow within a given API. A Test should be able to specify chains that it is capable of consuming.
• Purpose: To create more complex interactions with APIs, we must have some notion of state between requests, and enough knowledge about the contents of each response to pass onto the next request in the chain
• Example: A set of POST/GET/PUT/DELETE Request Templates for creating, retrieving, updating, and deleting a user in Keystone. These templates must somehow specify a way to retrieve the user's ID after creating it with the POST, and then pass it in the appropriate place(s) for the subsequent retrieval, updating, and deletion requests.

Scoring System

Signals

0 <= x < 0.5   -> INACTIVE
0.5 <= x < 1   -> PARTIALLY ACTIVE
x == 1         -> FULLY ACTIVE

Test Confidence

0 <= x < 5    -> LOW
5 <= x < 10   -> MEDIUM
x == 10       -> HIGH (10 = CEILING)