deeTEEcee

Two things we can do with C and Python integration.

1. C Extensions for Python

   In other words, we can write python methods in pure C.

2. Embedding Python a

The reasons for doing #1 are obvious because we can make speed improvements. For #2, it is due to transporting Python libraries over to C.

deeTEEcee

"""
How to compare 2 csv files such that I can identify:
* Added/Modified/Removed Rows but know specifically which header changed

There are two ways to look at diffs:
1. Line-by-line diffs

Line-by-line diffs are dumb and can't tell what a "Modified" item is.

2. Diffs with primary keys.

deeTEEcee

import re
text_list = [
    "P1:field1",
    "P1:field2"
    "P2:field1"
    "P2:field2",
    "P3:field1",
    "P3:field2",
    "P4:field1",
    "P5:field1",

deeTEEcee

-- Get Postgres Summary
--
create or replace function 
count_rows(schema text, tablename text) returns integer
as
$body$
declare
  result integer;
  query varchar;
begin

deeTEEcee

# HELP go_gc_duration_seconds A summary of the GC invocation durations.
# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 0.000244
go_gc_duration_seconds{quantile="0.25"} 0.000316201
go_gc_duration_seconds{quantile="0.5"} 0.0003472
go_gc_duration_seconds{quantile="0.75"} 0.000413601
go_gc_duration_seconds{quantile="1"} 0.010038708
go_gc_duration_seconds_sum 0.740180276
go_gc_duration_seconds_count 1456
# HELP go_goroutines Number of goroutines that currently exist.

deeTEEcee

Hex-style bit mapping

Because bytes tend to be longer, we'll want to use hex style mapping to represent bits. We know that hexadecimal is base 16 and bits are base 2 which translates to 4 bits = 1 hex.

Here's an example of why we want this case. If you have 16 bits, it looks like: 000011111111 but 0xFF is easier to read.

Here's a list of mapping for wanting only 1 bits: 0 -> 0x0 1 -> 0x1

deeTEEcee

# rs.initiate() is playing 2 roles:
# * required for my local mongo setup
# * it can also be used to test a connection but you can use almost any other evaluation command. 

# The command below runs a mongo check in a loop and times out after a minute.
# Why not use "wait-for-it.sh"? That establishes a tcp connection check but you could still run into a connection error
# if you run a single mongo connection check so we still need to loop.
timeout -v 1m bash -c 'until docker exec -it mongo mongo --eval "rs.initiate()"; do sleep 3; done'

deeTEEcee

Setting up database for testing

1. To build the image (since not hosted on dockerhub), follow the instructions on this page
   • If the dockerfile is incorrect, it's probably them just failing at README's. Try docker build -f Dockerfile.xe -t oracle/database:11.2.0.2-xe .
   • The README also provides more info. Under that page, search for the section referring to this specific image.
2. Do a test run: docker run -d -e ORACLE_PWD=oracle --shm-size="1g" -p 1521:1521 -p 8080:8080 oracle/database:11.2.0.2-xe (More details on --shm-size: oracle/docker-images#458)
   • The default database name will be 'xe.'
3. Add cx_Oracle / sqlalchemy to your system. Example here

Client installation (Debi

deeTEEcee

How to onboard a new engineer?

• Show general architecture through diagrams and concrete examples.
• There should be clear enough instructions that a new onboarding engineer can just look through a document and follow that to setup a local development environment and be able to set breakpoints and play around with the environment. (Even startups with smaller code bases will fail to follow up on this because it's not "priority" but it definitely should be.)

deeTEEcee

List of Difficult Bugs

The Gradle bug Case: I ran a basic gradle command ./gradlew help to test if the configuration and setup is done right. After I added a new plugin, I get an error saying a java method doesn't exist. How long did it take to solve: 1-2 days Reason it was unsolveable: A poor implementation of gradle has their dependencies leaking from one place to the other but it's really difficult to make that connection. Post-mortem: Nothing I really could've done here to speed up my effort. You need a deeper understanding of gradle internals (or to have used buildSrc before)