Skip to content

Instantly share code, notes, and snippets.

View ciberado's full-sized avatar

Javier Moreno ciberado

View GitHub Profile
@hboylan
hboylan / lambda-s3-read-write-by-line.js
Last active January 6, 2023 18:38
AWS Lambda function to read and write S3 files by line to perform efficient processing
const stream = require('stream')
const readline = require('readline')
const AWS = require('aws-sdk')
const S3 = new AWS.S3()
// read S3 file by line
function createReadline(Bucket, Key) {
// s3 read stream
const input = S3
@pecigonzalo
pecigonzalo / docker-dind-swarm-cluster.sh
Created June 9, 2017 17:32
Docker in Docker (dind) swarm creation script
#!/bin/bash
# set -x
MANAGER=${1:-3}
WORKER=${2:-0}
#=========================
# Creating cluster members
#=========================
@geuis
geuis / gist:8b1b2ea57d7f9a9ae22f80d4fbf5b97f
Last active January 10, 2024 16:43
Get Youtube video urls
// Run from the dev tools console of any Youtube video
// Accurate as of July 2, 2020.
//
// Copy and paste this into the dev console in a browser with the desired video loaded.
//
// NOTE: Some Youtube videos do not directly expose the video url in the response.
// This script doesn't currently attempt to handle those. It will work for most other general video types though.
(async () => {
const html = await fetch(window.location.href).then((resp) => resp.text()).then((text) => text);
@gene1wood
gene1wood / all_aws_managed_policies.json
Last active April 4, 2024 18:11
A list of all AWS managed policies and they're policy documents as well as a short script to generate the list
This file has been truncated, but you can view the full file.
{
"APIGatewayServiceRolePolicy": {
"Arn": "arn:aws:iam::aws:policy/aws-service-role/APIGatewayServiceRolePolicy",
"AttachmentCount": 0,
"CreateDate": "2019-10-22T18:22:01+00:00",
"DefaultVersionId": "v6",
"Document": {
"Statement": [
{
"Action": [
@acolyer
acolyer / service-checklist.md
Last active January 30, 2024 17:39
Internet Scale Services Checklist

Internet Scale Services Checklist

A checklist for designing and developing internet scale services, inspired by James Hamilton's 2007 paper "On Desgining and Deploying Internet-Scale Services."

Basic tenets

  • Does the design expect failures to happen regularly and handle them gracefully?
  • Have we kept things as simple as possible?