Skip to content

Instantly share code, notes, and snippets.

View tinovyatkin's full-sized avatar
💭
I may be slow to respond.

Konstantin Vyatkin tinovyatkin

💭
I may be slow to respond.
View GitHub Profile
@michelmilezzi
michelmilezzi / aws_glue_avoiding_duplicates.py
Last active August 29, 2023 22:08
AWS Glue script showing how to avoid duplicates during a job execution.
import sys
import pydevd
from awsglue.transforms import *
from awsglue.utils import getResolvedOptions
from pyspark.context import SparkContext
from awsglue.context import GlueContext
from awsglue.dynamicframe import DynamicFrame
from pyspark.sql.functions import col
from awsglue.job import Job
@outofcoffee
outofcoffee / find-ecr-image.sh
Last active March 1, 2024 13:35
Check if Docker image exists with tag in AWS ECR
#!/usr/bin/env bash
# Example:
# ./find-ecr-image.sh foo/bar mytag
if [[ $# -lt 2 ]]; then
echo "Usage: $( basename $0 ) <repository-name> <image-tag>"
exit 1
fi
IMAGE_META="$( aws ecr describe-images --repository-name=$1 --image-ids=imageTag=$2 2> /dev/null )"
@ebidel
ebidel / feature_detect_es_modules.js
Last active September 4, 2023 13:56
Feature detect ES modules: both static import and dynamic import()
<!--
Complete feature detection for ES modules. Covers:
1. Static import: import * from './foo.js';
2. Dynamic import(): import('./foo.js').then(module => {...});
Demo: http://jsbin.com/tilisaledu/1/edit?html,output
Thanks to @_gsathya, @kevincennis, @rauschma, @malyw for the help.
-->
@jwo
jwo / query.graphql
Last active November 26, 2020 15:11
GraphQL Example for GitHub API repo statuses on a pull request. We know before hand that it's "jwo/react-hover-image" and pull request number 4
{
repository(owner: "jwo", name:"react-hover-image"){
url
pullRequest(number: 4){
number
url
author {
avatarUrl
login
resourcePath
@maxbeatty
maxbeatty / lambda.js
Created August 22, 2017 20:01
using node-postgres (`pg`) in AWS Lambda
import λ from "apex.js";
import { Pool } from "pg";
// connection details inherited from environment
const pool = new Pool({
max: 1,
min: 0,
idleTimeoutMillis: 120000,
connectionTimeoutMillis: 10000
});

Scaling your API with rate limiters

The following are examples of the four types rate limiters discussed in the accompanying blog post. In the examples below I've used pseudocode-like Ruby, so if you're unfamiliar with Ruby you should be able to easily translate this approach to other languages. Complete examples in Ruby are also provided later in this gist.

In most cases you'll want all these examples to be classes, but I've used simple functions here to keep the code samples brief.

Request rate limiter

This uses a basic token bucket algorithm and relies on the fact that Redis scripts execute atomically. No other operations can run between fetching the count and writing the new count.

@paulirish
paulirish / server-timing-demo.js
Last active January 14, 2024 13:22
Demo of server timing values. visualized in chrome devtools
// see for screenshot:
// https://twitter.com/paul_irish/status/829090506084749312
const http = require('http');
function requestHandler(request, response) {
const headers = {
'Server-Timing': `
sql-1;desc="MySQL lookup Server";dur=100,
sql-2;dur=900;desc="MySQL shard Server #1",
@benjamincharity
benjamincharity / circle.yml
Last active April 23, 2019 14:43
CircleCI deploy to NPM
machine:
node:
version: 6.9.5
dependencies:
pre:
- 'echo "//registry.npmjs.org/:_authToken=${NPM_TOKEN}" > ~/.npmrc'
test:
override:
- rm -rf $CIRCLE_ARTIFACTS/coverage
@davisford
davisford / Setup MongoDB on localhost as Replica Set
Last active March 6, 2024 21:40
Setup MongoDB replica set on local host with only a single primary
Add the `replication` section to the mongod.conf file:
```
$cat /usr/local/etc/mongod.conf
systemLog:
destination: file
path: /usr/local/var/log/mongodb/mongo.log
logAppend: true
storage:
engine: mmapv1
@aaronshaf
aaronshaf / generator.js
Last active May 13, 2022 11:00
Use async generators and async iterators with DynamoDB's scan
const { docClient } = require('../services/dynamodb')
exports.findAllItems = async function* () {
let response = {}
let ExclusiveStartKey
do {
response = await docClient.scan({
TableName: 'mytable',
Limit: 500,
ExclusiveStartKey