Skip to content

Instantly share code, notes, and snippets.

View slaughtr's full-sized avatar

Dallas slaughtr

  • PNW
View GitHub Profile
@slaughtr
slaughtr / helpfulasyncaws.js
Last active April 25, 2018 20:51
Helpful async/await for AWS
// get a JSON file from S3 and immediately parse the JSON to a JS object
const testObj = JSON.parse(await s3.getObject({ Bucket: 'test', Key: test }).promise().then(res => res.Body.toString()));
// get all keys from an S3 listObjectV2 as an array (assuming no pagination of results)
const keysArr = (await s3.listObjectsV2({Bucket: 'test', Prefix: 'test'}).promise().then(res => res.Contents)).map(item => item.Key);

Keybase proof

I hereby claim:

  • I am slaughtr on github.
  • I am slaughtr (https://keybase.io/slaughtr) on keybase.
  • I have a public key ASAiAKcK3e9FF20mD_8CwhJBPrEz7apKVc2iiP-7DX3_DAo

To claim this, I am signing this object:

@slaughtr
slaughtr / twitter.js
Last active December 13, 2019 20:29
Twitter JS Scraper
const fetch = require('node-fetch');
const cheerio = require('cheerio');
const getTweets = async username => {
const req = await fetch(`https://twitter.com/${username}`);
const page = await req.text();
const $ = cheerio.load(page, {xmlMode: true});
const actualTweets = [];
const tweetHTML = $('.tweet-text').toArray();
@slaughtr
slaughtr / listAllBucketObjectsNoPrefixes.js
Created February 28, 2020 20:43
List all objects in S3 bucket (without prefixes)
const getAllObjects = async s3Params => {
const AWS = require('aws-sdk');
const S3 = new AWS.S3();
let results = [];
let gotAllObjs = false;
// Loop until we get all keys - This does NOT do any retries or backoff.
while (!gotAllObjs) {
if (!s3Params.ContinuationToken || s3Params.ContinuationToken !== 'DONE') {
const s3listres = await S3.listObjectsV2(s3Params).promise();
// assign ContinuationToken to data.NextContinuationToken

Keybase proof

I hereby claim:

  • I am slaughtr on github.
  • I am slaughtr (https://keybase.io/slaughtr) on keybase.
  • I have a public key ASC6lKQTnHAr3roSDE5Slqdcdv0i8bNH7JfXLFYHGbiGxQo

To claim this, I am signing this object:

@slaughtr
slaughtr / apacheRedirectCheck.js
Last active October 20, 2021 21:52
Check RewriteRules - find redirect chains, status codes, final URLs
// USAGE: node apacheRedirectCheck.js /path/to/file.conf.in https://yourdomain.example
const lineReader = require('line-by-line')
const util = require('util');
const exec = util.promisify(require('child_process').exec);
// get our file
const [ ,, fileName, host ] = process.argv
const lr = new lineReader(fileName)
const results = { success: [], regex: [], failure: []}
// Linereader boilerplate
lr.on('error', console.error)
@slaughtr
slaughtr / docker_config_var_mac.md
Created January 4, 2022 01:29
Docker for Mac - use DOCKER_CONFIG var

Docker for Mac does not respect the DOCKER_CONFIG var - docker/for-mac#2635

However, using launchctl you can get the desktop app to use a different .docker location.

  • Create a plist file, I use environment.plist as I use this workaround for other apps.
  • Insert the following into the plist file:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
@slaughtr
slaughtr / pre-commit-terraform.md
Last active January 5, 2022 07:14
pre-commit terraform
repos:
  - repo: https://github.com/antonbabenko/pre-commit-terraform
    rev: v1.55.0 # verify this is latest version
    hooks:
      - id: terraform_fmt
      - id: terraform_docs
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v2.4.0 # Use the ref you want to point at - check for updates
 hooks:
// I cannot absolutely guarantee I got everything completely correct, but I'm 95% sure I got at least 95% of it correct
// -----------------
// --- PROMISES
// -----------------
// Callback hell:
doSomething(params, (err, data) => {
if (err) console.error(err)
else {