Skip to content

Instantly share code, notes, and snippets.

Why Git Media

Git does not deal well with bunches of huge binary files, for a few reasons. It kills memory to add and commit them, it is pretty inefficient to transfer them, and it basically forces you to download all of them with each clone.

I thought sparse and narrow checkouts would be the answer, but it looks like it may be too difficult to rewrite other tools to deal with the missing blobs before they are fetched, as well as it still taking server resources to transfer them.

Perhaps a better way would be to hack the Git client slightly to store binary files over a certain size via a protocol that is better for that, like SFTP or HTTP over S3, and then fetch them only when necessary. Instead of the whole blob, just keep a blob that contains a pointer to the URL and SHA of the original content, or possibly just a SHA of the original content, since we could probably depend on the URL being guessable.

This is actually relatively similar to how git submodules does it - not ke

@sr
sr / gist:91275
Created April 7, 2009 15:07
Hax integrity to refresh the page
#!/usr/bin/env ruby
require "rubygems"
require "integrity"
class Refresher
JS = <<-EOS
<script type="text/javascript">
//<![CDATA[
setTimeout('location.reload()', %s * 1000)
//]]>
@sr
sr / gist:108184
Created May 7, 2009 16:08
Crazy test/unit + rack/test hax
require "test/unit"
require "rack/test"
require "contest"
require "sinatra/base"
class MyApp < Sinatra::Base
set :environment, :test
get "/" do
"Hi, #{params["name"]}."
@bitbckt
bitbckt / resque.rb
Created January 20, 2010 21:32
munin plugin for resque
#!/usr/bin/env ruby
require 'rubygems'
require 'resque'
HOST = ENV['host'] ? ENV['host'] : '127.0.0.1'
PORT = ENV['port'] ? ENV['port'] : '6379'
Resque.redis = "#{HOST}:#{PORT}"
2 Pac - Tradin War Stories
2 Pac - Holla At Me
50 Cent - Patiently Waiting
50 Cent - Heat
50 Cent - Back Down
50 Cent - Da Repercussions
50 Cent - Power Of The Dollar
G-Unit - Straight Outta Southside
G-Unit - Beg For Mercy
G-Unit - Gangsta Shit
build:
@echo "Combining files ..."
@cat \
js/dep/jquery.easing.js \
js/dep/jquery.jsonp.js \
js/dep/json2.js \
js/dep/toolbox.expose.js \
js/lib/jquery.transloadit2.js > build/jquery.transloadit2.js
@echo "Compiling with Closure REST API ..."
@curl \
@mjwall
mjwall / screen.rb
Created October 14, 2011 14:05
Homebrew screen install with 256 colors and vertical splits
require 'formula'
class Screen <Formula
homepage 'http://www.gnu.org/software/screen/'
url 'http://ftp.gnu.org/gnu/screen/screen-4.0.3.tar.gz'
md5 '8506fd205028a96c741e4037de6e3c42'
# brew install --HEAD to get the latests, includes vertical split, mapped
# to C-a |
head 'git://git.savannah.gnu.org/screen.git', :branch => 'master'
@halorgium
halorgium / github.com.js
Created November 1, 2011 16:10
Load the ISSUES?.md file content as the default issue body
// A crazy dotjs hack
// https://twitter.com/johnbender/status/131123843812179968
// You need to generate an OAuth token: http://developer.github.com/v3/oauth/#oauth-authorizations-api
var functionToLocation = function (fn) {
$(function () {
var js = fn.toString().replace(/\n/g, " ");
console.log(js);
window.location = "javascript: (" + js + ")()";
});
@dawud
dawud / recipe.rb
Created November 23, 2015 22:18
Tarsnap fpm-cookery recipe
class TarSnap < FPM::Cookery::Recipe
source 'https://www.tarsnap.com/download/tarsnap-autoconf-1.0.36.1.tgz'
homepage 'https://www.tarsnap.com/'
# Verify the signed SHA256 hash file from
# https://www.tarsnap.com/download/tarsnap-sigs-1.0.36.1.asc
sha256 'a2909e01e2f983179d63ef2094c42102c92c716032864e66ef25ae341ea28690'
name 'tarsnap'
version '1.0.36.1'
license 'Tarsnap License'
section 'tools'
@Visgean
Visgean / backup.py
Created July 16, 2012 22:27
Backup all messages on facebook
#! /usr/bin/python
# -*- coding: UTF-8 -*-
import facebook
import urllib2
import codecs
print "You need API token: get one here: https://developers.facebook.com/tools/explorer"
USER_ACCESS_TOKEN= raw_input("Your API key: ")