Skip to content

Instantly share code, notes, and snippets.

Making impact

Mark McDonnell Integralist

Block or report user

Report or block Integralist

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
Integralist / Golang ReverseProxy ErrorHandler Example.go
Last active May 21, 2019
[Golang ReverseProxy ErrorHandler Example] #go #golang #reverseproxy #error #redirect #302
View Golang ReverseProxy ErrorHandler Example.go
package main
import (
Integralist / Explain How Vary HTTP header
Last active May 22, 2019
[How Vary HTTP header works] #fastly #vcl #cdn #vary
View Explain How Vary HTTP header

Vary Behaviour

To understand HTTP's Vary behaviour we first must understand how the CDN works in the traditional sense of caching and looking up resources.

Note: at the bottom of this page is a sequence diagram which visually illustrates how Vary works (if you prefer a more visual aid).

The CDN caches requests based on a given pair of Host and Path values. So when the CDN accepts a request, as an example, for the resource the CDN will take the Host (e.g. and the Path (e.g. /videos) and generate a hash of those values which will become the 'cache key'. Later when the CDN receives the same request from a different client it'll generate the same hash and lookup the resource in its cache using the hash as the key.

Depending on the client, the origin server that generates response content (which will be cached in the CDN) may well want to serve different content for different clients

Integralist / 1. Code.go
Created May 9, 2019
[Go Unit Test Example] #go #golang #tests #testing #unittest
View 1. Code.go
// Split slices s into all substrings separated by sep and
// returns a slice of the substrings between those separators.
func Split(s, sep string) []string {
var result []string
i := strings.Index(s, sep)
for i > -1 {
result = append(result, s[:i])
s = s[i+len(sep):]
i = strings.Index(s, sep)
Integralist / AWS Security Group Rule - Ingress vs
Last active May 9, 2019
[AWS Security Group Rule - Ingress vs Egress] #aws #security #sg #sgr #rule
View AWS Security Group Rule - Ingress vs

The ever wonderful gave me the following analogy to help me understand the difference between ingress and egress with regards to defining them on 'rules' assigned to Security Groups.

My understanding was helped here when I started thinking of ingress and egress as two separate doors to a building.

Ingress needs to be told where things are coming from, even if they're coming from the service, because they are entering the "in" door, which faces an open street. Specifying the source on these rules is like having an ID badge that lets you through that in door, instead of having to send you to the desk. Even if the request comes from "inside the service", to the perspective of the service it's coming in the in door and so explicitly needs to know whether it has a badge (i.e. is from the 'source' security group).

Egress on the other hand is an out door. The service already knows who is "inside" the building, and it's just trying to send them ou

Integralist / Python Poetry.bash
Last active May 7, 2019
[Python Poetry] #python3 #poetry
View Python Poetry.bash
# install
curl -sSL | python
# reload .bash_profile and check poetry version
poetry --version
# update poetry to latest version
poetry self:update
# generate auto-complete for Homebrew installed version of bash
Integralist / Python3 Stream
Created May 7, 2019
[Python3 Stream Server] #python3 #asyncio #stream #server
View Python3 Stream
import asyncio
import contextvars
client_addr_var = contextvars.ContextVar('client_addr') # type:ignore
def render_goodbye():
# The address of the currently handled client can be accessed
# without passing it explicitly to this function.
Integralist / CDN Logs Sampling
Created Apr 26, 2019
[CDN Logs Sampling Rates] #cdn #logs #sampling #maths #requests #traffic #load
View CDN Logs Sampling

CDN Uncached Request Log Sampling


When we began passing Fastly logs through to Datadog, we were concerned with the volume of requests that we'd be logging. To cut down on this volume we decided to only pass through requests that were not satisfied by the cache. While this has reduced the log volume to a great degree, at times of high traffic we've seen millions of log events still passed through. To remain cost conscious while using Datadog to get visibility into these logs a sampling strategy should be implemented.

Proposed Sampling Rates

Fastly Status Meaning of Status Sample Rate Justification
Integralist / Symlinking.bash
Created Apr 23, 2019
[Symlinking] #terminal #command #symlink
View Symlinking.bash
# ln -s <real_path> <fake_path>
# for example, if we want ./lib to point to the real directory in ../../lib/go
ln -s ../../lib/go ./lib
Integralist /
Last active Apr 17, 2019
[Fastly Create Users + API tokens for Auditing purposes] #fastly #cli #auth #api

Create a new user (using an API token that has 'superuser' permissions):

curl -v -H "Fastly-Key: $FASTLY_API_TOKEN_SUPERUSER" -X POST -d "name=Foo Bar&"

Response (notice no 2FA, no password, no force password reset etc):

Integralist / Fastly VCL Multiplication.vcl
Created Apr 16, 2019
[Fastly VCL Multiplication] #fastly #vcl #cdn #multiplication
View Fastly VCL Multiplication.vcl
// generate nanoseconds from microseconds
declare local var.start-nano INTEGER;
set var.start-nano = std.strtol(time.start.usec, 10);
set var.start-nano *= 1000;
log var.start-nano;
set req.http.X-Start-in-Nanoseconds = var.start-nano;
You can’t perform that action at this time.