Skip to content

Instantly share code, notes, and snippets.

View dkam's full-sized avatar

Dan Milne dkam

View GitHub Profile
@dkam
dkam / favicon.rb
Last active December 15, 2023 07:51
Get a site's favicon
class Favicon
def initialize(doc, url: nil)
@doc = doc
@url = if url
@url = URI(url.to_s)
elsif canonical
URI(canonical)
else
@dkam
dkam / gist:37d7a41d3c4ab00278580531832130bd
Last active November 27, 2023 02:07
Postgres table, index & toast size.
# This ( https://gist.github.com/hatarist/675dc3debf6cf5f825b5c15aed4cbac0 ) with TOAST size formatting
SELECT
table_name,
pg_size_pretty(table_bytes) AS table,
pg_size_pretty(index_bytes) AS index,
pg_size_pretty(toast_bytes) AS toast,
pg_size_pretty(total_bytes) AS total
FROM (
SELECT
@dkam
dkam / songlink.rb
Created October 31, 2023 02:41
Lookup albums on song.link for sharing
@dkam
dkam / add_query_parameters_to_a_url.rb
Created September 15, 2023 05:15
Add query parameter with URI
URI(url).tap {|u| u.query = URI.encode_www_form(CGI.parse(u.query || "").merge({ref: 123})) }
@dkam
dkam / misc.rb
Created September 5, 2023 06:08
Logging in rails
# Make add tags to the normal rails logger, accessed like `hc_logger.info("my log message")`
def hc_logger = @hc_logger ||= logger.tagged("HttpClients").tagged(self.class.name)
# Create a new logger as an instance variable for a class:
class MyServiceClass
# ....
def my_logger = @my_logger ||= create_logger.tagged(self.class.name)
@dkam
dkam / bot_shedding.rb
Last active August 10, 2023 02:06
A Rails before_action to request bots retry-after X seconds when normalised load is high
def bot_shedding(wait: 600, threshold: 1)
require "sys/cpu"
require "etc"
# normalize the 1-minute load average based on processor count
return unless Sys::CPU.load_avg[0] / Etc.nprocessors > threshold
return unless DeviceDetector.new(request.user_agent).bot?
logger.info("Shedding bots: Returning 503 / Retry-After: #{wait} for #{request.remote_ip} / #{request.user_agent}")
/\b(?:97[89]-?)?(?:\d{1,5}-?)?(?:\d{1,7}-?)?(?:\d{1,6}-?)?\d{1,3}\b/
@dkam
dkam / oshash.rb
Created June 21, 2023 03:17
Open Subtitles Hash in Ruby
# https://trac.opensubtitles.org/projects/opensubtitles/wiki/HashSourceCodes#Ruby
# https://github.com/trytriangles/oshash/tree/master/test
def oshash(filename)
mask = 0xffffffffffffffff
readsize = 64 * 1024
filesize = File.size(filename)
hash = filesize
File.open(filename, 'rb') do |f|
hash = f.read(readsize).unpack("Q<*").inject(hash) {|h, v| h + v & mask}
@dkam
dkam / free.rb
Created May 19, 2023 04:10
Get free disk space
#!/usr/bin/env ruby
require 'bundler/inline'
gemfile do
source 'https://rubygems.org'
gem 'json'
gem 'sys-filesystem'
end
@dkam
dkam / valid_json.rb
Last active April 23, 2023 07:43
Check JSON is valid
def valid_json?(string)
result = JSON.parse(string)
result.is_a?(Hash) || result.is_a?(Array)
rescue JSON::ParserError, TypeError
false
end
def valid_json(string, default: [])
result = JSON.parse(string)