tomdottom

Basic task processing with distributed rate limiting

Proof of concept of distributed rate limiting multiple workers processing speed.

Rate limiting follows a leaky bucket algorithim. The bucket is implemented using a speical token-bucket queue. Max size of the bucket is enforced by using the max length of a token queue. The bucket is refilled by a single worker, which also is responsible for the refill rate.

Workers must get a token before fetching and processing any tasks.

Remi-C

DROP FUNCTION IF EXISTS find_useless_postgres_file(text);
CREATE OR REPLACE FUNCTION find_useless_postgres_file(    database_name text) 
	RETURNS TABLE(file_name text, file_relative_path text,file_abs_path text,size bigint) AS 
	$BODY$
		DECLARE     
			_useless record; 
		BEGIN  

			RETURN QUERY

paulirish

// http://paulirish.com/2011/requestanimationframe-for-smart-animating/
// http://my.opera.com/emoller/blog/2011/12/20/requestanimationframe-for-smart-er-animating

// requestAnimationFrame polyfill by Erik Möller. fixes from Paul Irish and Tino Zijdel

// MIT license

(function() {
    var lastTime = 0;
    var vendors = ['ms', 'moz', 'webkit', 'o'];