Skip to content

Instantly share code, notes, and snippets.

View sbailliez's full-sized avatar

Stephane B. sbailliez

View GitHub Profile
@sbailliez
sbailliez / gist:11268423
Created April 24, 2014 20:27
Scalyr RDS Postgresql log parser
{
lineGroupers: [
{
// groups line used to display stats
start: "^[^\\s]",
continueThrough: "^[\\s]+!"
},
{
// try to group lines that do have queries with text data containing newlines. Over simplification.
start: "^\\d+-\\d+-\\d+\\s+\\d+:\\d+:\\d+ UTC",
Customer customer = new Customer();
.... save customer ...
Card card = new Card(cardHref);
if (card.customer == null) {
card.associateToCustomer(customer);
}
Debit debit = card.debit(...);
DebitResult result = from(debit);
@sbailliez
sbailliez / gist:0a741dc5e21b42ef6736
Created March 24, 2015 14:21
index on element of json array ?
SELECT so._id AS order_id,
oi.value ->> 'id'::text AS id,
(oi.value -> 'sku_id'::text) ->> '$oid'::text AS sku_id,
FROM orders so,
LATERAL jsonb_array_elements(so._extra_props -> 'order_items'::text) oi(value);
@sbailliez
sbailliez / gist:8876660
Created February 8, 2014 04:20
Provision vagrant with ssh keys, pem file, git config, aws config, etc...
#
# Allows copying ssh keys, directly to vagrant.. quick hack for my needs
#
# It is basically reading the file, encoding it in base64, pass it in the shell, decode it and write it
# in the file with the same name in /home/vagrant, chown it and set 600 permissions
def provision_home(config, from, to = nil, user = 'vagrant', permission = "600")
if File.exists?(File.join(Dir.home, from))
require "base64"
to = to.nil? ? "/home/vagrant/#{from}" : to
require 'etc'
ssh_user = Etc.getlogin
public_key_path = "~/.ssh/id_dsa.pub"
ssh_public_key = File.read(File.expand_path(public_key_path)).strip
Vagrant.configure("2") do |config|
....
config.vm.provision :shell, :inline => "cat >>/tmp/ssh_key_pub <<EOF\n#{ssh_public_key}\nEOF"
config.vm.provision :shell, :inline => "cat >>/tmp/ssh_user <<EOF\n#{ssh_user}\nEOF"

Keybase proof

I hereby claim:

  • I am sbailliez on github.
  • I am sbailliez (https://keybase.io/sbailliez) on keybase.
  • I have a public key ASBKBaDbi1k-vE33JKl3XDKxxhj-K6OANQ1lIJ0WRNZjMwo

To claim this, I am signing this object:

@sbailliez
sbailliez / postgresql_11_upgrade.md
Last active May 7, 2021 14:11
Upgrading from Postgresql 10.6 to Postgresql 11.1

Spent a few hours dealing with errors on RDS when upgrading from a postgresql 10.6 to 11.1. In case someone else is dealing with the same problems, this might be helpful.

A few things to know about this database:

  • PostGIS extension was installed but was not being used
  • ip4r extension was installed but was also not used

This probably turned out to be a Good Thing (tm). If you are using PostGIS extensively, I wish you good luck.

Upgrades

@sbailliez
sbailliez / vagrant-vmware-no-network.log
Created November 20, 2022 04:29
Log file showing failure to run vmrun getGuestIPAddress
This file has been truncated, but you can view the full file.
[11/19/2022, 11:19:07.391 PM] INFO global: Vagrant version: 2.3.2
[11/19/2022, 11:19:07.391 PM] INFO global: Ruby version: 2.7.6
[11/19/2022, 11:19:07.391 PM] INFO global: RubyGems version: 3.1.6
[11/19/2022, 11:19:07.391 PM] INFO global: VAGRANT_LOG="debug"
[11/19/2022, 11:19:07.393 PM] INFO global: VAGRANT_EXECUTABLE="/opt/vagrant/embedded/gems/2.3.2/gems/vagrant-2.3.2/bin/vagrant"
[11/19/2022, 11:19:07.393 PM] INFO global: VAGRANT_INSTALLER_ENV="1"
[11/19/2022, 11:19:07.393 PM] INFO global: VAGRANT_INSTALLER_EMBEDDED_DIR="/opt/vagrant/embedded"
[11/19/2022, 11:19:07.393 PM] INFO global: VAGRANT_INSTALLER_VERSION="2"
[11/19/2022, 11:19:07.393 PM] WARN global: resolv replacement has not been enabled!
@sbailliez
sbailliez / compress.sql
Created June 1, 2024 19:11
dbt macro (redshift only) to compress a specified set of columns
{#
Compresses the columns of a table to the specified encodings. Unlike the compress_table macro in dbt-redshift,
you have to manually specify the encoding of the columns you want to compress, it does not run and use
the result of `ANALYZE compression <table>` which would is extremely slow to run every time
on a large table, especially when there is nothing to do and the table is already compressed.
If the specified column does not exist or if the specified encoding is identical to the current encoding,
the column is skipped.
ALTER statements are generated for each column where encoding needs to be changed.
@sbailliez
sbailliez / table_columns.sql
Created June 17, 2024 22:06
Redshift - Returns all columns datatype, encoding, nullable and default value. Useful for maintenance
WITH table_columns AS (
SELECT
c.relowner as tableowner
,n.nspname AS schema_name
,c.relname AS table_name
,a.attnum AS ordinal
,QUOTE_IDENT(a.attname) AS column_name
,CASE WHEN STRPOS(UPPER(format_type(a.atttypid, a.atttypmod)), 'CHARACTER VARYING') > 0
THEN REPLACE(UPPER(format_type(a.atttypid, a.atttypmod)), 'CHARACTER VARYING', 'VARCHAR')
WHEN STRPOS(UPPER(format_type(a.atttypid, a.atttypmod)), 'CHARACTER') > 0