Skip to content

Instantly share code, notes, and snippets.

🎯
Making impact

Mark McDonnell Integralist

Block or report user

Report or block Integralist

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@Integralist
Integralist / Golang ReverseProxy ErrorHandler Example.go
Last active May 21, 2019
[Golang ReverseProxy ErrorHandler Example] #go #golang #reverseproxy #error #redirect #302
View Golang ReverseProxy ErrorHandler Example.go
// https://play.golang.org/p/pk1FOh563jJ
package main
import (
"errors"
"fmt"
"io/ioutil"
"log"
"net"
@Integralist
Integralist / Explain How Vary HTTP header works.md
Last active May 22, 2019
[How Vary HTTP header works] #fastly #vcl #cdn #vary
View Explain How Vary HTTP header works.md

Vary Behaviour

To understand HTTP's Vary behaviour we first must understand how the CDN works in the traditional sense of caching and looking up resources.

Note: at the bottom of this page is a sequence diagram which visually illustrates how Vary works (if you prefer a more visual aid).

The CDN caches requests based on a given pair of Host and Path values. So when the CDN accepts a request, as an example, for the resource https://www.buzzfeed.com/videos the CDN will take the Host (e.g. www.buzzfeed.com) and the Path (e.g. /videos) and generate a hash of those values which will become the 'cache key'. Later when the CDN receives the same request from a different client it'll generate the same hash and lookup the resource in its cache using the hash as the key.

Depending on the client, the origin server that generates response content (which will be cached in the CDN) may well want to serve different content for different clients

@Integralist
Integralist / 1. Code.go
Created May 9, 2019
[Go Unit Test Example] #go #golang #tests #testing #unittest
View 1. Code.go
// Split slices s into all substrings separated by sep and
// returns a slice of the substrings between those separators.
func Split(s, sep string) []string {
var result []string
i := strings.Index(s, sep)
for i > -1 {
result = append(result, s[:i])
s = s[i+len(sep):]
i = strings.Index(s, sep)
}
@Integralist
Integralist / AWS Security Group Rule - Ingress vs Egress.md
Last active May 9, 2019
[AWS Security Group Rule - Ingress vs Egress] #aws #security #sg #sgr #rule
View AWS Security Group Rule - Ingress vs Egress.md

The ever wonderful twitter.com/sweetpavement gave me the following analogy to help me understand the difference between ingress and egress with regards to defining them on 'rules' assigned to Security Groups.


My understanding was helped here when I started thinking of ingress and egress as two separate doors to a building.

Ingress needs to be told where things are coming from, even if they're coming from the service, because they are entering the "in" door, which faces an open street. Specifying the source on these rules is like having an ID badge that lets you through that in door, instead of having to send you to the desk. Even if the request comes from "inside the service", to the perspective of the service it's coming in the in door and so explicitly needs to know whether it has a badge (i.e. is from the 'source' security group).

Egress on the other hand is an out door. The service already knows who is "inside" the building, and it's just trying to send them ou

@Integralist
Integralist / Python Poetry.bash
Last active May 7, 2019
[Python Poetry] #python3 #poetry
View Python Poetry.bash
# install
curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py | python
# reload .bash_profile and check poetry version
poetry --version
# update poetry to latest version
poetry self:update
# generate auto-complete for Homebrew installed version of bash
@Integralist
Integralist / Python3 Stream Server.py
Created May 7, 2019
[Python3 Stream Server] #python3 #asyncio #stream #server
View Python3 Stream Server.py
import asyncio
import contextvars
client_addr_var = contextvars.ContextVar('client_addr') # type:ignore
def render_goodbye():
# The address of the currently handled client can be accessed
# without passing it explicitly to this function.
@Integralist
Integralist / CDN Logs Sampling Rates.md
Created Apr 26, 2019
[CDN Logs Sampling Rates] #cdn #logs #sampling #maths #requests #traffic #load
View CDN Logs Sampling Rates.md

CDN Uncached Request Log Sampling

Why?

When we began passing Fastly logs through to Datadog, we were concerned with the volume of requests that we'd be logging. To cut down on this volume we decided to only pass through requests that were not satisfied by the cache. While this has reduced the log volume to a great degree, at times of high traffic we've seen millions of log events still passed through. To remain cost conscious while using Datadog to get visibility into these logs a sampling strategy should be implemented.

Proposed Sampling Rates

Fastly Status Meaning of Status Sample Rate Justification
@Integralist
Integralist / Symlinking.bash
Created Apr 23, 2019
[Symlinking] #terminal #command #symlink
View Symlinking.bash
# ln -s <real_path> <fake_path>
# for example, if we want ./lib to point to the real directory in ../../lib/go
ln -s ../../lib/go ./lib
@Integralist
Integralist / Example.md
Last active Apr 17, 2019
[Fastly Create Users + API tokens for Auditing purposes] #fastly #cli #auth #api
View Example.md

Create a new user (using an API token that has 'superuser' permissions):

curl -v -H "Fastly-Key: $FASTLY_API_TOKEN_SUPERUSER" -X POST -d "name=Foo Bar&login=foo.bar%40example.com" https://api.fastly.com/user

Response (notice no 2FA, no password, no force password reset etc):

@Integralist
Integralist / Fastly VCL Multiplication.vcl
Created Apr 16, 2019
[Fastly VCL Multiplication] #fastly #vcl #cdn #multiplication
View Fastly VCL Multiplication.vcl
// generate nanoseconds from microseconds
declare local var.start-nano INTEGER;
set var.start-nano = std.strtol(time.start.usec, 10);
set var.start-nano *= 1000;
log var.start-nano;
set req.http.X-Start-in-Nanoseconds = var.start-nano;
You can’t perform that action at this time.