deeTEEcee

Two things we can do with C and Python integration.

1. C Extensions for Python

   In other words, we can write python methods in pure C.

2. Embedding Python a

The reasons for doing #1 are obvious because we can make speed improvements. For #2, it is due to transporting Python libraries over to C.

deeTEEcee

"""
How to compare 2 csv files such that I can identify:
* Added/Modified/Removed Rows but know specifically which header changed

There are two ways to look at diffs:
1. Line-by-line diffs

Line-by-line diffs are dumb and can't tell what a "Modified" item is.

2. Diffs with primary keys.

deeTEEcee

import re
text_list = [
    "P1:field1",
    "P1:field2"
    "P2:field1"
    "P2:field2",
    "P3:field1",
    "P3:field2",
    "P4:field1",
    "P5:field1",

deeTEEcee

-- Get Postgres Summary
--
create or replace function 
count_rows(schema text, tablename text) returns integer
as
$body$
declare
  result integer;
  query varchar;
begin

deeTEEcee

# HELP go_gc_duration_seconds A summary of the GC invocation durations.
# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 0.000244
go_gc_duration_seconds{quantile="0.25"} 0.000316201
go_gc_duration_seconds{quantile="0.5"} 0.0003472
go_gc_duration_seconds{quantile="0.75"} 0.000413601
go_gc_duration_seconds{quantile="1"} 0.010038708
go_gc_duration_seconds_sum 0.740180276
go_gc_duration_seconds_count 1456
# HELP go_goroutines Number of goroutines that currently exist.

deeTEEcee

Setting up database for testing

1. To build the image (since not hosted on dockerhub), follow the instructions on this page
   • If the dockerfile is incorrect, it's probably them just failing at README's. Try docker build -f Dockerfile.xe -t oracle/database:11.2.0.2-xe .
   • The README also provides more info. Under that page, search for the section referring to this specific image.
2. Do a test run: docker run -d -e ORACLE_PWD=oracle --shm-size="1g" -p 1521:1521 -p 8080:8080 oracle/database:11.2.0.2-xe (More details on --shm-size: oracle/docker-images#458)
   • The default database name will be 'xe.'
3. Add cx_Oracle / sqlalchemy to your system. Example here

Client installation (Debi

deeTEEcee

Hex-style bit mapping

Because bytes tend to be longer, we'll want to use hex style mapping to represent bits. We know that hexadecimal is base 16 and bits are base 2 which translates to 4 bits = 1 hex.

Here's an example of why we want this case. If you have 16 bits, it looks like: 000011111111 but 0xFF is easier to read.

Here's a list of mapping for wanting only 1 bits: 0 -> 0x0 1 -> 0x1

deeTEEcee

# rs.initiate() is playing 2 roles:
# * required for my local mongo setup
# * it can also be used to test a connection but you can use almost any other evaluation command. 

# The command below runs a mongo check in a loop and times out after a minute.
# Why not use "wait-for-it.sh"? That establishes a tcp connection check but you could still run into a connection error
# if you run a single mongo connection check so we still need to loop.
timeout -v 1m bash -c 'until docker exec -it mongo mongo --eval "rs.initiate()"; do sleep 3; done'

deeTEEcee

### Databases
* One does not need to add database indexes for every field. I think one way of thinking about it is use the ones that people are querying that don't also already have indexes. If there is a multi-field query that's commonly used and there's already indexes, you probably don't need to add another index for a new field you add since that query already narrows it down for you. (There's a lot more to read to understand this) 

#### Internal
* SSD's vs Hard Disk
  * 
  * Writes have problems on both sides.  
* Access Patterns - Sequential vs. Random

Links

deeTEEcee

package com.sightmachine.doclet;

import java.lang.reflect.Array;
import java.lang.reflect.InvocationHandler;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;
import java.util.ArrayList;
import java.util.List;