sgoley

{
  "$schema": "https://raw.githubusercontent.com/JanDeDobbeleer/oh-my-posh/main/themes/schema.json",
  "blocks": [
    {
      "alignment": "left",
      "segments": [
        {
          "type": "text",
          "style": "diamond",
          "leading_diamond": "\ue0b6",

sgoley

# GitHub Action that uses sqlfmt to reformat the sql in an incoming pull request.
# If all sql in the pull request is compliant with sqlfmt then this Action does nothing.
# Otherwise, sqlfmt is run and its changes are committed back to the incoming pull request.

name: fmtdeploy
on: [pull_request]
jobs:
  fmt:
    runs-on: ubuntu-latest
    steps:

sgoley

#!/bin/bash

NOW=$(date "+%Y-%m-%d")
mkdir $HOME/tmp/envs-$NOW
ENVS=$(conda env list | grep '^\w' | cut -d' ' -f1)
for env in $ENVS; do
    source activate $env
    conda env export > $HOME/tmp/envs-$NOW/$env.yml
    echo "Exporting $env"
done

sgoley

{
    "python.defaultInterpreterPath": ".venv/scripts/python.exe"
}

sgoley

=LOWER(CONCATENATE(DEC2HEX(RANDBETWEEN(0;POWER(16;8));8);"-";DEC2HEX(RANDBETWEEN(0;POWER(16;4));4);"-";"4";DEC2HEX(RANDBETWEEN(0;POWER(16;3));3);"-";DEC2HEX(RANDBETWEEN(8;11));DEC2HEX(RANDBETWEEN(0;POWER(16;3));3);"-";DEC2HEX(RANDBETWEEN(0;POWER(16;8));8);DEC2HEX(RANDBETWEEN(0;POWER(16;4));4)))

sgoley

Running vacuum and analyze in dbtCloud

dbt and dbtCloud have the ability to run regular Redshift maintenance jobs. It's great to set these up early on in a project so that things stay clean as the project grows, and implementing these jobs in dbtCloud allows the same easy transparency and notifications as with your other dbt jobs.

This document will go through the specific steps necessary to configure vacuum and analyze jobs in the current version of dbt and dbtCloud. In the future, there will likely be a more idiomatically consistent way to express this logic using native dbt operations. Currently, this does work even if it is not elegant.

Step 1: Create the macros

macros/redshift_maintenance.sql

sgoley

-- in schema as procedure:

create function manage_magic_timestamp_columns() returns trigger
    strict
    cost 1
    language plpgsql
as
$$
BEGIN
    -- is a new record?

sgoley

-- author: Derek Sivers (https://github.com/sivers)
-- description: simple ideas / concepts / tagging table layout for PostgreSQL

CREATE TABLE concepts (
id integer primary key,
created_at date not null default CURRENT_DATE,
title varchar(127) not null unique CONSTRAINT title_not_empty CHECK (length(concepts."title") > 0),
concept text not null unique CONSTRAINT concept_not_empty CHECK (length(concepts."concept") > 0)
);
CREATE TABLE urls (

sgoley

{% macro grant_select_on_schemas(schemas, user) %}
  {% for schema in schemas %}
    grant usage on schema {{ schema }} to "{{ user }}";
    grant select on all tables in schema {{ schema }} to "{{ user }}";
    alter default privileges in schema {{ schema }}
        grant select on tables to "{{ user }}";
  {% endfor %}
{% endmacro %}

sgoley

# For Windows users# Note: <> denotes changes to be made

#Create a conda environment
conda create --name <environment-name> python=<version:2.7/3.5>

#To create a requirements.txt file:
conda list #Gives you list of packages used for the environment

conda list -e > requirements.txt #Save all the info about packages to your folder