Skip to content

Instantly share code, notes, and snippets.

Avatar
👯‍♀️
OMG

Ian Vaughan IanVaughan

👯‍♀️
OMG
View GitHub Profile
View runner.log
2021-06-08T11:59:41.3519967Z ##[section]Starting: Request a runner to run this job
2021-06-08T11:59:42.1598461Z Can't find any online and idle self-hosted runner in current repository that matches the required labels: 'ubuntu-latest'
2021-06-08T11:59:42.1598552Z Can't find any online and idle self-hosted runner in current repository's account/organization that matches the required labels: 'ubuntu-latest'
2021-06-08T11:59:42.1598921Z Found online and idle hosted runner in current repository's account/organization that matches the required labels: 'ubuntu-latest'
2021-06-08T11:59:42.3122458Z ##[section]Finishing: Request a runner to run this job
2021-06-08T11:59:51.7098919Z Current runner version: '2.278.0'
2021-06-08T11:59:51.7127239Z ##[group]Operating System
2021-06-08T11:59:51.7128229Z Ubuntu
2021-06-08T11:59:51.7128778Z 20.04.2
2021-06-08T11:59:51.7129289Z LTS
@IanVaughan
IanVaughan / Heroku grok logstash rule for Heroku Deployments.md
Last active Jan 20, 2021
Heroku grok logstash rule for Heroku Deployments
View Heroku grok logstash rule for Heroku Deployments.md

Sample:

Deploy 77bd8aca by user some@email.com

Rule:

rule1 Deploy %{notSpace:deploy.sha} by user %{data:deploy.user}
@IanVaughan
IanVaughan / Datadog grok logstash rules for standard sidekiq logs.md
Last active Jan 20, 2021
Datadog grok logstash rules for standard sidekiq logs
View Datadog grok logstash rules for standard sidekiq logs.md

Sample:

I, [2020-04-05T20:10:03.013349 #4]  INFO -- : [Paypal::Transactions] jid:39d5996400fa48a, too account_id:3, end_date:2020-04-04T23:59:59Z, page:1

Rules:

sidekiq %{notSpace:log.letter},\s+\[%{date("yyyy-MM-dd'T'HH:mm:ss.SSSSSS"):log.date}\s+#.\]\s+%{word:log.level}\s+--\s+\:\s+\[%{notSpace:sidekiq.job.class}]\s+jid:%{word:sidekiq.job.jid}%{data::keyvalue(":", "", ",")}
@IanVaughan
IanVaughan / Datadog grok logstash rules for custom sidekiq logs.md
Last active Jan 20, 2021
Datadog grok logstash rules for custom sidekiq logs
View Datadog grok logstash rules for custom sidekiq logs.md

Samples:

I, [2020-11-12T12:12:18.883446 #4]  INFO -- : sidekiq-stats {"processed":48469869,"failed":30091,"scheduled_size":0,"retry_size":0,"dead_size":0,"processes_size":1,"default_queue_latency":0,"workers_size":0,"enqueued":0}
I, [2020-11-12T12:03:11.112090 #4]  INFO -- : sidekiq-queues [[{"name":"default"},{"count":0},{"size":0},{"latency":0}],[{"name":"import"},{"count":0},{"size":0},{"latency":0}],[{"name":"low"},{"count":26},{"size":26},{"latency":183.94694018363953}]]

Rules:

autoFilledRule1 I,\s+\[%{date("yyyy-MM-dd'T'HH:mm:ss.SSSSSS"):date}\s+#4\]\s+%{word:level}\s+--\s+\:\s+sidekiq-stats\s+\{\"processed\"\:%{integer:processed},\"failed\"\:%{integer:failed},\"scheduled_size\"\:%{integer:scheduled_size},\"retry_size\"\:%{integer:retry_size},\"dead_size\"\:%{integer:dead_size},\"processes_size\"\:%{integer:processes_size},\"default_queue_latency\"\:%{number:default_queue_latency},\"workers_size\"\:%{integer:workers_size},\"enqueued\"\:%{integer:enqueued}\}
autoFilledRule2 I,\s+\[%{date("yy
@IanVaughan
IanVaughan / gist:b604d5b26a0b8fa36c4fab9188cdb6db
Created Sep 18, 2020
failed to build gem native extension. xcode
View gist:b604d5b26a0b8fa36c4fab9188cdb6db
Fetching nio4r 2.5.4
Installing nio4r 2.5.4 with native extensions
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
current directory: /Users/ianvaughan/.asdf/installs/ruby/2.7.1/lib/ruby/gems/2.7.0/gems/nio4r-2.5.4/ext/nio4r
/Users/ianvaughan/.asdf/installs/ruby/2.7.1/bin/ruby -I /Users/ianvaughan/.asdf/installs/ruby/2.7.1/lib/ruby/2.7.0 -r ./siteconf20200918-40223-cj3k1e.rb extconf.rb
checking for unistd.h... *** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.
@IanVaughan
IanVaughan / hash-to-json.rb
Created Sep 15, 2020
Convert a log of hashes into json files in ruby
View hash-to-json.rb
i=0
file = File.read('log')
file.each_line do |line|
hash = eval line
File.write("data-#{i}.json", hash.to_json)
puts i
i+=1
end
@IanVaughan
IanVaughan / upgrade_db.sh
Created Aug 27, 2020
Steps to Upgrade a Heroku DB in a script
View upgrade_db.sh
#!/usr/bin/env bash
# https://devcenter.heroku.com/articles/updating-heroku-postgres-databases#updating-with-follower-changeover
APP=name-production # name-staging
heroku addons:create heroku-postgresql:standard-2 --follow DATABASE_URL --app $APP # DONE
heroku addons:wait # DONE
for (( ; ; )); do; heroku pg:info --app $APP | grep Behind; sleep 1; done # DONE
@IanVaughan
IanVaughan / gist:9ac9a30597719b3e86d6b252e0db9c92
Created Aug 27, 2020
Intresting differences between rails routing form_for helpers
View gist:9ac9a30597719b3e86d6b252e0db9c92
# [:admin, @application] => admin_onboarding_url
# [:admin, @collections_facility] => admin_collections_facility_url
# [:admin, @application, @collections_facility] => admin_onboarding_collections_facility_url
# [:admin, :application, @collections_facility] => No route matches {:action=>"show", :application_id=>#<Collections::Facility id: 409, user_id: 1303, amount: 0.1e5, revenue_share: 0.25e1, fee: 0.31e1, created_at: "2020-01-24 10:45:50", updated_at: "2020-01-24 10:45:50", application_id: 886>, :controller=>"admin/collections/facilities"}, missing required keys: [:id]
# [:admin, :application, @application] => admin_application_onboarding_url
# format.html { redirect_to [:admin, :application, @application], notice: 'Facility was successfully created.' }
# redirect_to [:admin, :application, @collections_facility]
# => "<html><body>You are being <a href=\"http://test.host/admin/applications/233/collections/facilities/233\">redirected</a>.</body></html>"
@IanVaughan
IanVaughan / team_stats.rb
Created Aug 27, 2020
Find GitHub team PR counts
View team_stats.rb
# frozen_string_literal: true
# https://gist.github.com/borischerkasky/7acc8b19e7b5c6b044bf92ab6f467126
require 'awesome_print'
require 'httparty'
class TeamPrStatistics
attr_accessor :stats
@IanVaughan
IanVaughan / unused_routes.rb
Created Aug 27, 2020
Find unused rails routes
View unused_routes.rb
# frozen_string_literal: true
Rails.application.eager_load!
unused_routes = {}
# Iterating over all non-empty routes from RouteSet
Rails.application.routes.routes.map(&:requirements).reject(&:empty?).each do |route|
name = route[:controller].camelcase
next if name.start_with?('Rails')