This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
heroku config:set $(awk -F'=' '{print $1"="$2}' <env_file>) -a <app_name? |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
s3 = boto3.client('s3') | |
with open('FILE_NAME', 'wb') as f: | |
s3.download_fileobj('BUCKET_NAME', 'OBJECT_NAME', f) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from sklearn.compose import ColumnTransformer, make_column_selector | |
from sklearn.pipeline import Pipeline | |
col_name_pattern = "feature_" | |
pl = Pipeline([ | |
( | |
"col_selector", | |
ColumnTransformer([ | |
("selector", "passthrough", make_column_selector(pattern=col_name_pattern)) | |
], remainder="drop") |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
""" | |
The issue is that the unittest runner replaces sys.stdout/sys.stderr before the testing starts, and the StreamHandler is still writing to the original sys.stdout. | |
""" | |
import sys | |
import unittest | |
import logging | |
logger = logging.getLogger() | |
logger.level = logging.DEBUG |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
/* | |
If counts are more than one, then there are either duplicates or it is not the correct business key | |
This will also return the content of rows | |
*/ | |
select * | |
from ( | |
select * | |
, count(1) over ( | |
partition by <BK> | |
) as cn |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
select ordinal_position, | |
column_name, | |
data_type, | |
case when character_maximum_length is not null | |
then character_maximum_length | |
else numeric_precision end as max_length, | |
is_nullable, | |
column_default as default_value | |
from information_schema.columns | |
where table_name = 'batch_logger' -- enter table name here |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Get commands | |
kubectl get services # List all services in the namespace | |
kubectl get deployment my-dep # List a particular deployment | |
kubectl get replicaset # List all replica sets | |
kubectl get pods # List all pods in the namespace | |
kubectl get pods -o wide # List all pods in the current namespace, with more details | |
kubectl get pod my-pod -o yaml # Get a pod's YAML | |
# Describe commands | |
kubectl describe nodes my-node |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
SELECT r.rolname, r.rolsuper, r.rolinherit, | |
r.rolcreaterole, r.rolcreatedb, r.rolcanlogin, | |
r.rolconnlimit, r.rolvaliduntil, | |
ARRAY(SELECT b.rolname | |
FROM pg_catalog.pg_auth_members m | |
JOIN pg_catalog.pg_roles b ON (m.roleid = b.oid) | |
WHERE m.member = r.oid) as memberof | |
, r.rolreplication | |
, r.rolbypassrls | |
FROM pg_catalog.pg_roles r |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# if there is a MFA set up for the account | |
# you can use this script to assume a role | |
#!/bin/sh | |
export AWS_PROFILE=my_profile | |
ACCOUNT_ID=2128506 # production account id | |
TEST_ACCOUNT_ID=666420 | |
TEST_ACCOUNT_ROLE=testDataEngineerRole | |
USERNAME=bg |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
function range(stopAt, startAt = 0) { | |
return [...Array(stopAt).keys()].map(i => i + startAt); | |
} |
NewerOlder