Skip to content

Instantly share code, notes, and snippets.

View hrchu's full-sized avatar
:octocat:
Have an Octotastic day!

petertc hrchu

:octocat:
Have an Octotastic day!
View GitHub Profile
@hrchu
hrchu / gist:c964f18d60b44f0bb9af
Last active August 29, 2015 14:08
Emulate large file in python, with file-like object interface and without memory overload
file_size=5*2**30 #5GB
class FakeFile:
def __init__(self):
self.remain_size=file_size
def read(self, size):
if self.remain_size == 0:
return None
elif self.remain_size >= size:
self.remain_size-=size
@hrchu
hrchu / complete_mpu.py
Created November 12, 2014 05:13
Complete unfinished multipart upload by boto
import boto
from client import conn
bucket_name = '[YOUR BUCKET NAME]'
object_key = '[YOUR OBJECT KEY]'
upload_id = '[UPLOAD ID], can get it by s3cmd multipart s3://[YOUR BUCKET NAME]'
def main():
bucket = conn.lookup(bucket_name)
complete_part_upload(bucket, object_key, upload_id)

Example: You have a branch refactor that is quite different from master. You can't merge all of the commits, or even every hunk in any single commit or master will break, but you have made a lot of improvements there that you would like to bring over to master.

Note: This will not preserve the original change authors. Only use if necessary, or if you don't mind losing that information, or if you are only merging your own work.

On master:

> git co -b temp
@hrchu
hrchu / 0_reuse_code.js
Last active August 29, 2015 14:10
Here are some things you can do with Gists in GistBox.
// Use Gists to store code you would like to remember later on
console.log(window); // log the "window" object to the console
@hrchu
hrchu / python_resources.md
Created December 4, 2014 01:43 — forked from jookyboi/python_resources.md
Python-related modules and guides.

Packages

  • lxml - Pythonic binding for the C libraries libxml2 and libxslt.
  • boto - Python interface to Amazon Web Services
  • Django - Django is a high-level Python Web framework that encourages rapid development and clean, pragmatic design.
  • Fabric - Library and command-line tool for streamlining the use of SSH for application deployment or systems administration task.
  • PyMongo - Tools for working with MongoDB, and is the recommended way to work with MongoDB from Python.
  • Celery - Task queue to distribute work across threads or machines.
  • pytz - pytz brings the Olson tz database into Python. This library allows accurate and cross platform timezone calculations using Python 2.4 or higher.

Guides

@hrchu
hrchu / gist:33bbd35a4b95288ad695
Created March 18, 2015 09:18
yet another s3 policy example
{
"Version": "2008-10-17",
"Id": "Policy1426504761539",
"Statement": [
{
"Sid": "Stmt1426504756244",
"Effect": "Deny",
"Principal": {
"AWS": "*"
},
@hrchu
hrchu / gist:0cc5e783387f5453f528
Last active April 23, 2019 09:48
logstash-grok-pattern-squid
# Follows the squid format in default:
# logformat squid %ts.%03tu %6tr %>a %Ss/%03>Hs %<st %rm %ru %[un %Sh/%<a %mt
# http://www.squid-cache.org/Doc/config/logformat/
%{NUMBER:timestamp}\s+%{NUMBER:response_time} %{IPORHOST:src_ip} %{WORD:squid_request_status}/%{NUMBER:http_status_code} %{NUMBER:reply_size_include_header} %{WORD:http_method} %{URI:request_url} %{USERNAME:user} %{WORD:squid_hierarchy_status}/%{IPORHOST:server_ip_or_peer_name} (?<mime_content_type>\S+\/\S+)
@hrchu
hrchu / MogileOperator.java
Created July 21, 2015 05:45
Test Moji concurrency behavior
import fm.last.moji.MojiFile;
/**
* Created by developer on 4/30/15.
*/
class MogileOperator implements Runnable {
@Override
public void run() {
MojiFile rickRoll = TestMojiConcurrency.moji.getFile("rick-astley");
@hrchu
hrchu / xstartup
Created October 22, 2015 02:24
~/.vnc/xstartup for ubuntu 14.04 (fix 3D causing problem)
#!/bin/sh
# Uncomment the following two lines for normal desktop:
# unset SESSION_MANAGER
# exec /etc/X11/xinit/xinitrc
[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
xsetroot -solid grey
vncconfig -iconic &
@hrchu
hrchu / start-chrome.sh
Created January 26, 2016 08:38
start google chrome in a new session, and import profile from origin
#!/bin/bash
# remove ':' from screen id
PROFILE="session${DISPLAY#:}"
PROFILE_DIRECTORY="$HOME/.google-chrome/$PROFILE"
mkdir -p $PROFILE_DIRECTORY
# import profile (includes bookmark) from origin
if [ ! -d "$PROFILE_PATH/Default" ]; then
cp -r $HOME/.config/google-chrome/Default $PROFILE_DIRECTORY/Default;