alonisser

--ignore-directory=node_modules
--ignore-directory=venv

alonisser

# Create the read stream
dataitemsAppendDf =  spark.readStream.format("delta")\
 .option("maxFilesPerTrigger", 25)\
 .table(f"{database_name}.{table_name}")

# Initiate the write stream
dataitemsAppendDf.writeStream \
 .trigger(processingTime='15 seconds') \ # note, you can also have different modes here
 .option("checkpointLocation", f"{checkpoint_name}") \
 .foreachBatch(processRawStreamBatch) \ #Callback that would handle each batch

alonisser

developing

• vite as a webpack alternative

tools

• valtio - state proxy
• formik forms handling
• react query http queries (or other async with caching

Testing

• mirage.js
• cypress - check integration with mirage

alonisser

bootstrap.servers={{ kafka_cluster.boostrap_servers }}
ssl.endpoint.identification.algorithm=https
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="{{kafka_cluster.username}}" password="{{kafka_cluster.password}}";

alonisser

FROM python:3.6-alpine
# Opted for alpine to get a lean docker image as possible
RUN apk add --no-cache openssl

ENV DOCKERIZE_VERSION v0.6.1
RUN wget https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-alpine-linux-amd64-$DOCKERIZE_VERSION.tar.gz \
    && tar -C /usr/local/bin -xzvf dockerize-alpine-linux-amd64-$DOCKERIZE_VERSION.tar.gz \
    && rm dockerize-alpine-linux-amd64-$DOCKERIZE_VERSION.tar.gz

# Python deps for alpine

alonisser

#Broken somehow. need to fix
SELECT COUNT(*) as datname, datname, datid FROM pg_stat_activity GROUP BY datname;
#Works without id
SELECT COUNT(*) as datname, datname FROM pg_stat_activity GROUP BY datname;

More info:
http://chrismiles.info/systemsadmin/databases/articles/viewing-current-postgresql-queries/

https://support.pivotal.io/hc/en-us/articles/201595093-Script-Session-level-information

alonisser

sudo apt-add-repository ppa:ansible/ansible
sudo apt-get update
sudo apt-get install ansible
mkdir automation && cd automation
touch ansible.cfg && hosts
mkdir -pv roles/aws_monitoring/tasks/main.yml

alonisser

echo iptables-persistent iptables-persistent/autosave_v4 boolean true | sudo debconf-set-selections
echo iptables-persistent iptables-persistent/autosave_v4 boolean true | sudo debconf-set-selections

sudo apt-get -y install iptables-persistent

alonisser

{
  "title": "Nginx Web",
  "services": {
    "query": {
      "list": {
        "0": {
          "query": "response:[200 TO 299]",
          "alias": "OK",
          "color": "#7EB26D",
          "id": 0,

alonisser

cost of data structures

notes on redis memory usage - the previous is based on this post

redis memory optimization page

Interesting post by instagram engineering on this subject

The profiling tool: rdb tools