Integralist

• Service Level Indicator: what we’ve chosen to measure progress towards our goal. E.g., “Latency of a request.”

• Service Level Objective: the stated objective of the SLI – what we’re trying to accomplish for either ourselves or the customer. E.g., “99.5% of requests will be completed in 5ms.”

• Service Level Agreement: generally speaking, this is a contract explicitly stating the consequences of failing to achieve your defined SLOs. E.g., “If 99% of your system requests aren’t completed in 5ms, you get a refund.”

Integralist

# Note: doing this requires _a_ version of Go to be installed already (e.g. `brew install go`)

# setup
mkdir -p ~/code
cd ~/code

# get source
git clone https://go.googlesource.com/go
cd go
git tag

Integralist

apt-cache search openssl

Integralist

FROM python:3.6.3-slim

# Dependencies
# NGINX: pcre, zlib, openssl (+ libssl-dev), build-essential (for c compiler)
# RUN apt-get update && apt-get install -y wget libpcre3 libpcre3-dev zlib1g zlib1g-dev openssl libssl-dev build-essential
RUN apt-get update && apt-get install -y wget build-essential

# Check Python
RUN python3 --version && pip3 --version
COPY ./requirements.txt /tmp/

Integralist

# imagine you have multiple lines (non-deterministically generated):
#
# server 127.0.0.1; other stuff
# server 127.2.2.2:2222; other stuff
#
# to replace the second line and not the first, you can use the /.../! syntax
# this says "as long as the first pattern doesn't match, go ahead and try the substitution"
#
# sed -i '/some_pattern_to_ignore/! s/some_pattern_to_match/the_replacement_for_the_match/g' /nginx.conf

Integralist

def auth(*args, **kwargs):
    client = kwargs['client']

    def wrapper(handler_method):
        """input will be, for example, RequestHandler.get or RequestHandler.post"""

        @wraps(handler_method)
        async def request_handler_wrapper(*args, **kwargs):
            """input is the request handler class and params from tornado.routing.Rule"""

Integralist

Distributed Tracing is the process of tracking and analyzing what happens to a request (transaction) across all services it touches.

• Transaction: request
• Trace ID: unique request identifier
• Trace: entire transaction (across all services)
• Span: logical chunk of work in a given 'Trace'

Spans have parent-child relationships. A parent can have multiple children. The first span is the "root span". Each span has an ID and a pointer to its parent span ID.

Note: this information was found on the Nike engineering blog

Integralist

ECS Task has memory allocation of 500mb.

It's a multi-process tornado service with 4 processes.

Each process is reporting 200+ mb of RSS (resident) memory usage, which would mean total memory should be larger than 500mb.

But as processes share memory, the total RSS is likely to be the sum of the memory shared across all child processes + the "unique" memory used by each child process as they diverge from one another.

Integralist

The following is copied verbatim from https://www.kitchensoap.com/2012/10/25/on-being-a-senior-engineer/ -- this gist is merely a partial backup of that excellent article.

For my own thoughts on the subject (which I still agree with having now read the above link) see my post called "The Perfect Developer Qualities": https://www.integralist.co.uk/posts/the-perfect-developer-qualities/

Mature engineers seek out constructive criticism of their designs.

Every successful engineer I’ve met, upon finishing up a design or getting ready for a project, will continually ask their peers questions along the lines of:

• “What could I be missing?”
• “How will this not work?”

Integralist

The below code snippets demonstrate how to make synchronous code asynchronous using a threadpool (in this example it's specifically handled within a tornado application).

If you're using the requests http client, then this can be made asynchronous using the same threadpool technique, although it can be better to use use an external library (such as requests-futures) if you need to do something a little more complex. For example, being able to utilise the requests library's Session feature would need extra work, and so an external library can help with that.

There are other alternative libraries for working with the requests library too:

• requests-threads (uses the Twisted Framework)
• grequests (uses the Gevent Framework)

Notes: Greques