IanVaughan

ccloud kafka topic create --partitions 1 test-topic

ccloud kafka topic produce test-topic --value-format avro --schema /path/to/schema.json
{"myField":"test","myField2":1}
{"myField":"foo","myField2":2}

Finish Getting Started with Schema Registry

IanVaughan

2021-07-29T11:29:38.0007456Z ##[group]Starting datadog-agent service container
2021-07-29T11:29:38.0009102Z ##[command]/usr/bin/docker pull datadog/agent:latest
2021-07-29T11:29:39.4210209Z latest: Pulling from datadog/agent
2021-07-29T11:29:39.7656448Z 1127acbecdf7: Pulling fs layer
2021-07-29T11:29:41.8115266Z 1127acbecdf7: Verifying Checksum
2021-07-29T11:29:41.8115918Z 1127acbecdf7: Download complete
2021-07-29T11:29:50.9679094Z 1127acbecdf7: Pull complete
2021-07-29T11:29:50.9742704Z Digest: sha256:da43df11da9f6052d1a8691c87c5b1df60e79352f94c4ed7a53465e1a51dac04
2021-07-29T11:29:50.9761841Z Status: Downloaded newer image for datadog/agent:latest
2021-07-29T11:29:50.9798695Z docker.io/datadog/agent:latest

IanVaughan

#!/usr/bin/env bash
# Usage
# cred-diff <env> (<branch|ref>|master)
# eg
# cred-diff production master

VAR=$2
REF="${VAR:-master}"
git checkout $REF --quiet
printf .

IanVaughan

#!/usr/bin/env bash
# Usage:
#  diff-creds production

git checkout master --quiet
printf .
rails credentials:show --environment $1 > /tmp/prod
printf .
git checkout - --quiet
printf .

IanVaughan

2021-06-08T11:59:41.3519967Z ##[section]Starting: Request a runner to run this job
2021-06-08T11:59:42.1598461Z Can't find any online and idle self-hosted runner in current repository that matches the required labels: 'ubuntu-latest'
2021-06-08T11:59:42.1598552Z Can't find any online and idle self-hosted runner in current repository's account/organization that matches the required labels: 'ubuntu-latest'
2021-06-08T11:59:42.1598921Z Found online and idle hosted runner in current repository's account/organization that matches the required labels: 'ubuntu-latest'
2021-06-08T11:59:42.3122458Z ##[section]Finishing: Request a runner to run this job
2021-06-08T11:59:51.7098919Z Current runner version: '2.278.0'
2021-06-08T11:59:51.7127239Z ##[group]Operating System
2021-06-08T11:59:51.7128229Z Ubuntu
2021-06-08T11:59:51.7128778Z 20.04.2
2021-06-08T11:59:51.7129289Z LTS

IanVaughan

Sample:

Deploy 77bd8aca by user some@email.com

Rule:

rule1 Deploy %{notSpace:deploy.sha} by user %{data:deploy.user}

IanVaughan

Sample:

I, [2020-04-05T20:10:03.013349 #4]  INFO -- : [Paypal::Transactions] jid:39d5996400fa48a, too account_id:3, end_date:2020-04-04T23:59:59Z, page:1

Rules:

sidekiq %{notSpace:log.letter},\s+\[%{date("yyyy-MM-dd'T'HH:mm:ss.SSSSSS"):log.date}\s+#.\]\s+%{word:log.level}\s+--\s+\:\s+\[%{notSpace:sidekiq.job.class}]\s+jid:%{word:sidekiq.job.jid}%{data::keyvalue(":", "", ",")}

IanVaughan

Samples:

I, [2020-11-12T12:12:18.883446 #4]  INFO -- : sidekiq-stats {"processed":48469869,"failed":30091,"scheduled_size":0,"retry_size":0,"dead_size":0,"processes_size":1,"default_queue_latency":0,"workers_size":0,"enqueued":0}
I, [2020-11-12T12:03:11.112090 #4]  INFO -- : sidekiq-queues [[{"name":"default"},{"count":0},{"size":0},{"latency":0}],[{"name":"import"},{"count":0},{"size":0},{"latency":0}],[{"name":"low"},{"count":26},{"size":26},{"latency":183.94694018363953}]]

Rules:

autoFilledRule1 I,\s+\[%{date("yyyy-MM-dd'T'HH:mm:ss.SSSSSS"):date}\s+#4\]\s+%{word:level}\s+--\s+\:\s+sidekiq-stats\s+\{\"processed\"\:%{integer:processed},\"failed\"\:%{integer:failed},\"scheduled_size\"\:%{integer:scheduled_size},\"retry_size\"\:%{integer:retry_size},\"dead_size\"\:%{integer:dead_size},\"processes_size\"\:%{integer:processes_size},\"default_queue_latency\"\:%{number:default_queue_latency},\"workers_size\"\:%{integer:workers_size},\"enqueued\"\:%{integer:enqueued}\}
autoFilledRule2 I,\s+\[%{date("yy

IanVaughan

Fetching nio4r 2.5.4
Installing nio4r 2.5.4 with native extensions
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

    current directory: /Users/ianvaughan/.asdf/installs/ruby/2.7.1/lib/ruby/gems/2.7.0/gems/nio4r-2.5.4/ext/nio4r
/Users/ianvaughan/.asdf/installs/ruby/2.7.1/bin/ruby -I /Users/ianvaughan/.asdf/installs/ruby/2.7.1/lib/ruby/2.7.0 -r ./siteconf20200918-40223-cj3k1e.rb extconf.rb
checking for unistd.h... *** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

IanVaughan

i=0
file = File.read('log')
file.each_line do |line|
  hash = eval line
  File.write("data-#{i}.json", hash.to_json)
  puts i
  i+=1
end