KartikTalwar

The fastest remote directory rsync over ssh archival I can muster (40MB/s over 1gb NICs)

This creates an archive that does the following:

rsync (Everyone seems to like -z, but it is much slower for me)

• a: archive mode - rescursive, preserves owner, preserves permissions, preserves modification times, preserves group, copies symlinks as symlinks, preserves device files.
• H: preserves hard-links
• A: preserves ACLs

hummus

wget http://stedolan.github.io/jq/download/linux64/jq 
aws ec2 describe-instances --filters "Name=tag:Name,Values=$NAME" \
  "Name=instance-state-name,Values=running" \
  |  jq -r \
  ".Reservations[] | .Instances[] | .InstanceId" \

aws ec2 describe-volumes --filters \
  "Name=status,Values=available" \
  | jq -r ".Volumes[] | .VolumeId" \

ktuite

import boto.mturk.connection

sandbox_host = 'mechanicalturk.sandbox.amazonaws.com'
real_host = 'mechanicalturk.amazonaws.com'

mturk = boto.mturk.connection.MTurkConnection(
    aws_access_key_id = 'XXX',
    aws_secret_access_key = 'XXX',
    host = sandbox_host,
    debug = 1 # debug = 2 prints out all requests. but we'll just keep it at 1

arikfr

Setup

$ pip install click requests

Usage

$ python query_export.py --redash-url "https://app.redash.io/" --api-key ""

marians

This is an unofficial manual for the couchdb Python module I wish I had had.

Installation

pip install couchdb

Importing the module

hqingyi

<source>
  type tail
  path /var/log/haproxy.log
  pos_file /var/log/td-agent/haproxy.log.pos
  tag haproxy.http
  format /^(?<syslog_time>.+) (?<source_ip>.+) (?<ps>\w+)\[(?<pid>\d+)\]: (?<c_ip>[\w\.]+):(?<c_port>\d+) \[(?<time>.+)\] (?<f_end>[\w\.-]+) (?<b_end>[\w\.-]+)\/(?<b_server>[^ ]+) (?<tq>[-]?\d+)\/(?<tw>[-]?\d+)\/(?<tc>[-]?\d+)\/(?<tr>[-]?\d+)\/(?<tt>\d+) (?<status_code>\d+) (?<bytes_read>\d+) (?<captured_request_cookie>.+) (?<captured_response_cookie>.+) (?<termination_state>.+) (?<actconn>\d+)\/(?<feconn>\d+)\/(?<beconn>\d+)\/(?<srv_conn>\d+)\/(?<retries>\d+) (?<srv_queue>\d+)\/(?<backend_queue>\d+) \"(?<http_request>.+)\"$/
  time_format %d/%b/%Y:%H:%M:%S.%L
</source>

<match *.**>

bastman

Docker - How to cleanup (unused) resources

Once in a while, you may need to cleanup resources (containers, volumes, images, networks) ...

delete volumes

// see: https://github.com/chadoe/docker-cleanup-volumes

$ docker volume rm $(docker volume ls -qf dangling=true)

$ docker volume ls -qf dangling=true | xargs -r docker volume rm

Belgabor

Guide on how to create and set up your Django project with webpack, npm and ReactJS :)

Hopefully this will answer "How do I setup or start a Django project?" I was trying to set up my own website, and there was a lot to read, learn and digest! Therefore, I put this together which is a collection of all the guides/blog posts/articles that I found the most useful. At the end of this, you will have your barebones Django app configured and ready to start building :)

NOTE: This guide was built using Django 1.9.5, NodeJS 4+ with NPM 3+

1. Setting up your dev environment

rifki192

conf = {
  "sqs-access-key": "",
  "sqs-secret-key": "",
  "sqs-queue-src": "",
  "sqs-queue-dest": "",
  "sqs-region": "ap-southeast-1",
}

import boto.sqs
conn = boto.sqs.connect_to_region(

takeshixx

Running Ubuntu 16.04.1 armhf on Qemu

This is a writeup about how to install Ubuntu 16.04.1 Xenial Xerus for the 32-bit hard-float ARMv7 (armhf) architecture on a Qemu VM via Ubuntu netboot.

The setup will create a Ubuntu VM with LPAE extensions (generic-lpae) enabled. However, this writeup should also work for non-LPAE (generic) kernels.

The performance of the resulting VM is quite good, and it allows VMs with >1G ram (compared to 256M on versatilepb and 1G on versatile-a9/versatile-a15). It also supports virtio disks whereas versatile-a9/versatile-a15 only support SD cards via the -sd argument.

Get netboot files