OAuth support in Cloudprober HTTP probes

gistfile1.txt

OAuth is a widely used HTTP authentication mechanism these days. It will be nice to add some OAuth support to Cloudprober.
This feature has also been requested by the users:
https://github.com/google/cloudprober/issues/27

Main complexity in adding OAuth support is in managing the token itself: How do we get it -- from config or retrieve from 
somewhere at the runtime, how often to refresh it, etc.

Since OAuth tokens usually expire, it's not very useful to specify tokens in the config or even environment variable 
(environment variables are passed at the process at the start time). In the config, we should specify the token source
and how often to access that token source. For example, a config could look like this:

oauth_config {
  bearer_token {
    # Run the following command to get the bearer token
    cmd: 'cat /var/lib/access/token | cut -d2 -f:'
      
    # Refresh token every 300s. Set to 0 for no caching.
    refresh_interval_sec: 300 # refresh every 5 min.
  }
}

Options in token_source could be:

 * file  (read a file -- for example, this will work for default tokens on GCE, GKE and AWS environments)
 * cmd (command's output)
 * url (just access a URL)
 * gce_service_account (Get token from GCE metadata)
 * default_aws (AWS's default application credentials)
 
We should define this config in such a way that we can use same notation for non-probe parts too -- for example, for k8s
API server authentication while not running in the same cluster, etc.

Implementation:

* Common OAuth module:

There will be a common OAuth module:
cloudprober/common/oauth/.
                        /proto/config.proto
                        oauth.go
                        
It will likely have the following interface:

ts, err := oauth.TokenSource(config)
...

tok := ts.Token() [or tokHeader := ts.TokenHeader()]

ts.Token() will take care of caching the token and refreshing it whenever required.
 

Thanks for looking into this! For what it's worth, blackbox_exporter can use basic auth, a fixed string or a file, whereas default_gce would be the most helpful for me.

https://github.com/prometheus/blackbox_exporter/blob/master/CONFIGURATION.md:

# The HTTP basic authentication credentials for the targets.
  basic_auth:
    [ username: <string> ]
    [ password: <secret> ]

  # The bearer token for the targets.
  [ bearer_token: <secret> ]

  # The bearer token file for the targets.
  [ bearer_token_file: <filename> ]

Thanks for the feedback @drigz. I'll look into integrating basic_auth as well. With my current implementation, this is what an example configuration ends up looking like.

probe {
  type: HTTP
  name: "gcs_private"
  targets {
    host_names: "storage.googleapis.com"
  }

  http_probe {
    protocol: HTTPS
    relative_url: "/test-bbmc-private-1/obj"

    oauth_config {
      bearer_token{
        gce_service_account: "default"
      }
    }
  }

  validator {
    name: "cp"
    regex: "test_cloudprober"
  }
}