Skip to content

Instantly share code, notes, and snippets.

View vanleantking's full-sized avatar
🤣

Le Van vanleantking

🤣
  • U
  • Binh Thanh
View GitHub Profile
package mapreduce
func MapReduce(mapper func(interface{}, chan interface{}),
reducer func(chan interface{}, chan interface{}),
input chan interface{},
pool_size int) interface{}
{
reduce_input := make(chan interface{});
reduce_output := make(chan interface{});
worker_output := make(chan chan interface{}, pool_size);
@jwickett
jwickett / Multi-Threaded_Web_Crawler.py
Created December 22, 2009 06:32
A multi-threaded Web crawler implemented in Python
import threading, urllib, urlparse
from HTMLParser import HTMLParser
import sys
class LinkHTMLParser(HTMLParser):
A_TAG = "a"
HREF_ATTRIBUTE = "href"
def __init__(self):
self.links = []
@rantav
rantav / README.md
Created June 27, 2012 05:18
MongoDB increment or insert

In mongodb it's easy to make at upsert, meaning update-or-insert by calling

db.collection.update({criteria}, {updated fields}, true)

The third parameter means - insert a new document if the document doesn't exist yet. So, for example, the following will insert a new document for the user if there's no document for that user yet, and will update it if it already exists:

 db.users.update({user_id: '1234'}, {user_id: '1234', name: 'Ran'}, true)
@AlexandreAbraham
AlexandreAbraham / unsupervised_alt.py
Last active February 5, 2023 23:00
These are two implementations of the silhouette score. They are compatible with the scikit learn implementation but offers different drawbacks in term of complexity and memory usage. The slow version needs no memory but is painfully slow and should, I think, not be used. The second one is based on a block strategy: distance between samples and c…
""" Unsupervised evaluation metrics. """
# License: BSD Style.
from itertools import combinations
import numpy as np
from sklearn.utils import check_random_state
from sklearn.metrics.pairwise import distance_metrics
from sklearn.metrics.pairwise import pairwise_distances
@nicolashery
nicolashery / elasticsearch.md
Last active December 30, 2023 19:03
Elasticsearch: updating the mappings and settings of an existing index

Elasticsearch: updating the mappings and settings of an existing index

Note: This was written using elasticsearch 0.9.

Elasticsearch will automatically create an index (with basic settings and mappings) for you if you post a first document:

$ curl -X POST 'http://localhost:9200/thegame/weapons/1' -d \
'{
  "_id": 1,
@kethinov
kethinov / walksync.js
Created September 22, 2013 09:04
List all files in a directory in Node.js recursively in a synchronous fashion
// List all files in a directory in Node.js recursively in a synchronous fashion
var walkSync = function(dir, filelist) {
var fs = fs || require('fs'),
files = fs.readdirSync(dir);
filelist = filelist || [];
files.forEach(function(file) {
if (fs.statSync(dir + file).isDirectory()) {
filelist = walkSync(dir + file + '/', filelist);
}
else {
@Cifro
Cifro / bookmark.html
Last active November 27, 2018 06:03
FB Friends Ranking
<a href="javascript:(function()%7Bfunction%20creator(e,t,n)%7Bvar%20r=document.createElement(n);var%20i=document.createTextNode(t);r.appendChild(i);e.appendChild(r)%7Dfunction%20displayData(e)%7Bvar%20t=document.createElement(%22table%22);var%20n=document.createElement(%22thead%22);t.appendChild(n);var%20r=document.createElement(%22tr%22);creator(r,%22Name%22,%22th%22);creator(r,%22Score%22,%22th%22);n.appendChild(r);var%20s=document.createElement(%22tbody%22);t.appendChild(s);for(i=0;i&lt;e.length;i++)%7Bvar%20r=document.createElement(%22tr%22);creator(r,e%5Bi%5D%5B%22text%22%5D,%22td%22);creator(r,e%5Bi%5D%5B%22grammar_costs%22%5D%5B%22%7Buser%7D%22%5D,%22td%22);s.appendChild(r)%7Ddocument.body.innerHTML=%22%22;document.body.appendChild(t)%7Did=requireDynamic(%22Env%22).user;url=%22//www.facebook.com/ajax/typeahead/search/facebar/bootstrap/?viewer=%22+id+%22&amp;__a=1%22;x=new%20XMLHttpRequest;x.onreadystatechange=function()%7Bif(x.readyState==4&amp;&amp;x.status==200)%7Bsrr=JSON.parse(x.responseText.substr
@ardan-bkennedy
ardan-bkennedy / GoMgoSample-1.go
Last active February 27, 2021 08:31
Sample Go and MGO example
type (
// BuoyCondition contains information for an individual station.
BuoyCondition struct {
WindSpeed float64 `bson:"wind_speed_milehour"`
WindDirection int `bson:"wind_direction_degnorth"`
WindGust float64 `bson:"gust_wind_speed_milehour"`
}
// BuoyLocation contains the buoy's location.
BuoyLocation struct {
@jefftriplett
jefftriplett / tor.py
Last active May 22, 2023 09:10
Python Requests + Tor (Socks5)
"""
setup:
pip install requests
pip install requests[socks]
super helpful:
- http://packetforger.wordpress.com/2013/08/27/pythons-requests-module-with-socks-support-requesocks/
- http://docs.python-requests.org/en/master/user/advanced/#proxies
"""
@dz1984
dz1984 / utils.go
Created March 25, 2014 06:22
Utility Functions with Golang.
package utils
import (
"bufio"
"fmt"
"io/ioutil"
"os"
)
func IsExists(path string) bool {