A Ruby Thor script that will parse a Google Logging URL and render to console with `pull` or to a log file and opened in editor with `open`.

_about.md

A Ruby Thor script that will parse a Google Logging URL and render to console with pull or to a log file and opened in editor with open.

The default editor when ENV "EDITOR" is not set is VS CODE, so code or ENV["EDITOR"] should be in your ENV $PATH -- open doesn't work with VIM as-is since the script already has the console that vim will want to use.

Example use:

ruby logs.rb open "https://console.cloud.google.com/logs/viewer?advancedFilter=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22devops-foo%0Aresource.labels.location%3D%22us-central1%22%0Aresource.labels.cluster_name%3D%22ci-gke-cluster%22%0Aresource.labels.namespace_name%3D%22foobar%22%0Aresource.labels.pod_name%3D%22qa-9999-smzc5%22%0A&interval=NO_LIMIT&project=devops-foo"

To use:

1. With Ruby >= 2.3, install the Thor gem. gem install thor
2. place this file somewhere on your $PATH and give it a memorable name. (i.e. logs, gcloud_logs, logs.rb)
3. make it executable: chmod +x logs.rb or just invoke with Ruby as shown above
4. copy a URL for the log query you want to fetch and paste after the open or pull command
5. run it!

logs.rb

#!/usr/bin/env ruby

require 'thor'
require 'uri'
require 'cgi'
require 'json'
require 'time'
require 'fileutils'

class TweakedThor < Thor
  # avoids the unhelpful deprecation warning that is otherwise emitted.
  def self.exit_on_failure?
    true
  end
end

# Extracts the Advanced Filter parameters
class Filter
  attr_reader :params

  def initialize filter
    @filter = filter
    transform_filter_into_params
  end

  def as_query
    @params.map{|k,v| "#{k}=#{v}"}.join(" AND ")
  end

  def pod_name
    @params["resource.labels.pod_name"] || "unknown"
  end

  # given a filter string, renders into a Hash afer removing embedded quotes, 
  # splitting on new lines and then splitting each entry on the equal sign
  # 
  #   e.g.  "resource.type=\"k8s_container\"\nresource.labels.project_id=\"devops-foo\"
  #         => {"resource.type" => "k8s_container", "resource.labels.project_id" => "devops-foo"}
  def transform_filter_into_params
    @params = CGI.unescape(@filter)
      .gsub("\"","")
      .split("\n")
      .map{|p| p.split("=")}
      .to_h
  end
end

# Extracts the query parameters
class QueryParams
  attr_reader :query, :params

  def initialize query
    @query = query
    @params = query.split("&").map{|q| q.split("=")}.to_h
    validate!
  end

  def filter
    @filter ||= Filter.new(@params["advancedFilter"])
  end

  def validate!
    return unless Hash(@params).fetch("advancedFilter", {}).empty?
    message = "Cannot process #{@params.inspect}\n"
    message << "The query is missing the `advancedFilter` parameter!"
    raise RuntimeError, message
  end
end

# Transforms a JSON entry into human-readable string suitable for emitting to console or to log file
class Entry
  def initialize entry
    @entry = entry
  end

  def empty?
    @entry.nil? || @entry.fetch("textPayload", "").empty?
  end

  def as_time(value)
    ts = Time.parse value
    ts.strftime("%H:%M:%S.%L")
  end
  
  def to_s
    return "" if empty?

    output = "[%s] %s: %s" % [    
      as_time(@entry["timestamp"]), 
      @entry["severity"].rjust(5),
      @entry["textPayload"].chomp
    ]
  end
end

# Implements the tasks for pulling down logs from the Google Cloud
class LogCLI < TweakedThor
  desc "pull URL", "pulls the recent logs for the given URL"
  def pull url
    params = extract_params url
    fetch_entries(params).each{ |entry| puts entry }
  end

  desc "open URL", "pulls the recent logs for the given URL and opens in editor"
  def open url
    params = extract_params url
    entries = fetch_entries(params)
    timestamp = Time.now
    log_filename = File.join("log", [params.filter.pod_name, timestamp.strftime("%Y%m%d"), timestamp.to_i.to_s].join("-") << ".log")
    FileUtils.mkdir("log") unless File.exist?("log")
    File.open(log_filename, "w"){ |f| entries.each{ |e| f.puts e } }
    `#{env_editor} #{log_filename}`
  end

  private

  def env_editor
    ENV["EDITOR"] || "code"
  end

  # Google sometimes separates query params with semi-colons, which isn't URI compliant so 
  # Ruby's built-in parsing doesn't hold the query params in uri.query
  # We munge the uri.path so it looks like a typical `advancedFilter` query
  # ALERT: Total Hack!
  def munge_google_style_query uri
    return unless uri.path =~ /^\/logs\/query;query/
    uri.path.gsub(/^\/logs\/query;query/, 'advancedFilter').gsub(";", "&")
  end

  def extract_params url
    uri = URI(url)
    query = munge_google_style_query(uri) || uri.query
    QueryParams.new query
  end

  def fetch_entries params
    JSON.parse(gcloud_cmd params).map{ |payload| Entry.new(payload) }
  end

  def gcloud_cmd params
    cmd = %{gcloud logging read "#{params.filter.as_query}" --format=json}
    puts "*" * 80, cmd.gsub(" AND", "\n   AND "), "*" * 80
    `#{cmd}`
  end
end

LogCLI.start ARGV