Skip to content

Instantly share code, notes, and snippets.

@jamesbjackson
jamesbjackson / azure_storage_account.json
Last active December 3, 2019 10:45
Azure Support in Sparkleformation.io
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"name": {
"type": "string",
"defaultValue": "storage_account_name",
"metadata": {
"description": "Unique DNS name for the Storage Account"
}

Three system configuration parameters must be set to support a large number of open files and TCP connections with large bursts of messages. Changes can be made using the /etc/rc.d/rc.local or /etc/sysctl.conf script to preserve changes after reboot.

1. /proc/sys/fs/file-max: The maximum number of concurrently open files.

fs.file-max = 1000000

2. /proc/sys/net/ipv4/tcp_max_syn_backlog: Maximum number of remembered connection requests, which are still did not receive an acknowledgment from connecting client. The default value is 1024 for systems with more than 128Mb of memory, and 128 for low memory machines.

net.ipv4.tcp_max_syn_backlog = 3240000

3. /proc/sys/net/core/somaxconn: Limit of socket listen() backlog, known in userspace as SOMAXCONN. Defaults to 128.

net.core.somaxconn = 3240000

#!/bin/bash -x
#---------------------------------------------------------------------------------------------------
# Variables
#---------------------------------------------------------------------------------------------------
#Set the software versions to be downloaded and installed.
openresty_version="1.11.2.2"
#----------------------------------------------------------------------------------------------------------------------------

Rancher Single Server Install

Install Docker

sudo apt-get update
sudo apt-get -y install apt-transport-https ca-certificates curl software-properties-common 
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-get update
apt-cache policy docker-ce
  • Maximum Open Files
You requested maxclients of 10000 requiring at least 10032 max file descriptors.
Server can't set maximum open files to 10032 because of OS error: Operation not permitted.
Current maximum open files is 4096. maxclients has been reduced to 4064 to compensate for maxclients has been reduced to 4064 to compensate for low ulimit. If you need higher maxclients increase 'ulimit -n'.
@jamesbjackson
jamesbjackson / rate-limit.lua
Created July 31, 2019 11:11 — forked from jbaiter/rate-limit.lua
Simple Nginx Rate Limiting with Lua, redis and redis-cell
-- Requires the `redis-cell` module to be installed in Redis: https://github.com/brandur/redis-cell
local redis = require "nginx.redis"
local red = redis:new()
red:set_timeout(1000)
local ok, err = red:connect("127.0.0.1", 6379)
if not ok then
ngx.log(ngx.ERR, "failed to connect to redis: ", err)
ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)
@jamesbjackson
jamesbjackson / gist:6ef38d59761517dbf474b55fdcdf4f13
Created July 4, 2019 11:07 — forked from dantswain/gist:fdfb1c2c86e4d940a8f5
Convert Elixir config.exs to Erlang sys.config
#!/usr/bin/env elixir
# Convert an Elixir config.exs to an erlang sys.config
#
# Usage: elixir_to_sys_config config.exs > sys.config
# First argument is the Elixir config.exs file
# Writes to stdout
# You probably want to set MIX_ENV accordingly
#
# 2015 by Dan Swain, dan.t.swain@gmail.com
@jamesbjackson
jamesbjackson / update_cache.rake
Created July 3, 2019 15:55 — forked from apsoto/update_cache.rake
Sync Chef Cookbooks and Roles with Server
#
# Tasks to keep your repository in sync with your Chef Server
#
# Author:: Matthew Kent (<mkent@magoazul.com>)
# Copyright:: Copyright (c) 2010 Matthew Kent
# License:: Apache License, Version 2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
@jamesbjackson
jamesbjackson / README.md
Created May 21, 2019 13:15
How to show current HTTP requests on any server

How to show current HTTP requests on any server

via https://dev.to/codeenigma/how-to-show-current-http-requests-on-any-server-mmp

"If you’re ever need to see incoming HTTP requests, maybe to check incoming headers, you want to know what cookies are being set, or get some clues as to why things aren’t being cached, the following command might be helpful:"

sudo tcpdump -A -s 10240 'tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)' | egrep --line-buffered "........(GET |HTTP\/|POST |HEAD )|[A-Za-z0-9-]+: " | sed -r 's/........(GET |HTTP\/|POST |HEAD )/\n\1/g'

"It’s a bit of a mouthful, but it shows incoming HTTP requests direct from the network interface, and formats them in a readable way for humans. The nice thing is that you can then use grep with the output to show things like incoming cookies, or request headers. Probably more useful for monitoring requests than watching log files and guessing."

@jamesbjackson
jamesbjackson / Makefile
Created May 17, 2019 14:34 — forked from fawkesley/Makefile
Makefile for activating a virtualenv and installing requirements. Uses requirements-to-freeze.txt / requirements.txt pattern
# Put *unversioned* requirements in `requirements-to-freeze.txt` as described below.
# `requirements.txt` will be automatically generated from `pip freeze`
# https://www.kennethreitz.org/essays/a-better-pip-workflow
venv/bin/activate: requirements-to-freeze.txt
rm -rf venv/
test -f venv/bin/activate || virtualenv -p $(shell which python3) venv
. venv/bin/activate ;\
pip install -Ur requirements-to-freeze.txt ;\
pip freeze | sort > requirements.txt