bruno-uy

select s3l.message, s3l.*, sle.*
from stl_load_errors sle
left join svl_s3log s3l
    on sle.query = s3l.query
order by sle.starttime desc
limit 10;

bruno-uy

variable = "This is the value"
print(f"{variable=}")
# >>> variable='This is the value'

bruno-uy

-- This setting disables the results cache, so we can see the full processing runtime each time we run the query
SET enable_result_cache_for_session TO OFF;

bruno-uy

from inspect import currentframe, getframeinfo

print(getframeinfo(currentframe()).lineno)  # prints 3

bruno-uy

-- Problem: you don't see all the schemas when querying PG_TABLE_DEF
-- Solution:
-- 1. First check if the schema you're trying to query is on the search path
show search_path;
-- 2. Add the missing one(s) to the search path (imagine the result was only public and you're missing data_warehouse and matching)
set search_path to '$user', public, data_warehouse, matching;  -- No matter which is your user, use '$user'

bruno-uy

Git good practices

1. Write meaningful and concise commit message:
   • ❌ "Add new feature"
   • ✅ Changing X and Y because of Z
2. Follow a pattern / convention for commit messages. You can check a good reference here.
3. Squash commits you did for testing / adding small changes. You can check how to do that here.
4. Separate your commits into isolated units of "atomic" changes. Examples:
   • Changes in one class / file
   • Refactor previous to the actual change you'll be doing

• Changes in one function if the change is considerable

bruno-uy

Squash commits

Definition: combine multiple commits into one. More related to get tidy commits than a technical problem about not doing that.

You need to first figure out how many commits do you have to squash. To check that you can use:

git log

Imagine you wanna combine the last 3 commits into one. You'll do a soft reset from HEAD minus 3 commits:

bruno-uy

import pandas as pd

pd.set_option("display.max_rows", 500)
pd.set_option("display.max_columns", 500)
pd.set_option("display.width", 1000)
pd.set_option("display.max_colwidth", None)

bruno-uy

import pandas as pd
import numpy as np

df = pd.DataFrame({"A": [1, 2, 3], "B": [1.2, np.NaN, 3.4]})
result = (
    df
    .replace([np.nan], [None], regex=False)
    .to_dict(orient="records")
)

bruno-uy

import pandas as pd
df = pd.concat([pd.read_csv(f, compression="gzip") for f in os.listdir() if f.endswith(".gz")], ignore_index=True)