Skip to content

Instantly share code, notes, and snippets.

WooodHead

Block or report user

Report or block WooodHead

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@WooodHead
WooodHead / scrapy-installation.md
Created Aug 4, 2019 — forked from samiujan/scrapy-installation.md
How to install Scrapy on Ubuntu
View scrapy-installation.md

Scrapy is the web-scraper's scraper - it handles typical issues like distributed, asynchronous crawling, retrying during down-time, throttling download speeds, pagination, image downloads, generates beautiful logs and does much much more

You need a few modules to run scrapy on a Ubuntu/Debian machine (I used a cloud-based Ubuntu 14.04.4 LTS)

Following are the steps (and some recommendations)

The following was executed on a vanilla DigtialOcean Ubuntu (5 USD per month, 512 MB RAM). I feel this is sufficient to run a Scrapy crawler running at approx 1 HTTP request per second (with auto-throttle and delays turned on)

sudo apt-get update
@WooodHead
WooodHead / puppeteer-gif.js
Created May 22, 2019 — forked from bambooom/puppeteer-gif.js
use puppeteer to generate gif
View puppeteer-gif.js
const fs = require('fs');
const puppeteer = require('puppeteer');
const GIFEncoder = require('gifencoder');
const PNG = require('png-js');
function decode(png) {
return new Promise(r => {png.decode(pixels => r(pixels))});
}
@WooodHead
WooodHead / libreadline_6_not_found.sh
Created Mar 25, 2019 — forked from wbotelhos/libreadline_6_not_found.sh
Library not loaded: /usr/local/opt/readline/lib/libreadline.6.dylib (LoadError)
View libreadline_6_not_found.sh
ln -s /usr/local/opt/readline/lib/libreadline.7.0.dylib /usr/local/opt/readline/lib/libreadline.6.dylib
@WooodHead
WooodHead / aria2c_webUI.md
Created Mar 25, 2019 — forked from GAS85/aria2c_webUI.md
Aria2 + Ubuntu 16.04 + Apache2 + Web UI
View aria2c_webUI.md
OS: Ubuntu 16.04 Apache/2.4.18 1.0.2g-1ubuntu4.10
Aim: to install Aria2 with WebUI and secure Token.
IP Addr of your Aria2 server is 111.222.111.222
Your local IP network is 192.168.0.0/24

Aria 2

1. Installation

Install aria2 package:

@WooodHead
WooodHead / git-collaborative-workflow.md
Created Feb 13, 2019 — forked from adamloving/git-collaborative-workflow.md
Simple Git workflow for collaborating on a project. I wrote this to help a co-worker learn Git (and help me remember after a year of working on my own).
View git-collaborative-workflow.md

Creating the change

$ git checkout -b my-feature

... modify code ....

$ git add <filename> 
$ git commit -m “my feature is this”
View wget-ignore-robots.txt
$ wget -e robots=off --wait 1 http://your.site.here
View Mongodb restore and dump commands via json, bson and csv.md

NOTE -

  • Remove -h option if you are doing operation on same machine
  • Remove -u , -p option if your database don't have username and password

Binary

Import database

mongorestore -h IP:port -d DB_Name -u user_name -p password <input db directory>
View circular-json-stringify.js
function circularJSONStringify(obj) {
const cache = [];
const result = JSON.stringify(obj, (key, value) => {
if (typeof value === 'object' && value !== null) {
if (cache.indexOf(value) !== -1) {
// Circular reference found, discard key
return;
}
// Store value in our collection
cache.push(value);
View mouseevent.path for safari and firefox.js
if (!('path' in Event.prototype)) {
Object.defineProperty(Event.prototype, 'path', {
get: function () {
const path = [];
let currentElem = this.target;
while (currentElem) {
path.push(currentElem);
currentElem = currentElem.parentElement;
}
if (path.indexOf(window) === -1 && path.indexOf(document) === -1)
@WooodHead
WooodHead / install-and-configure-shadowsocks-server.sh
Created Oct 14, 2018 — forked from euyuil/install-and-configure-shadowsocks-server.sh
Shadowsocks: Server installation and configuration on Ubuntu 14.04.
View install-and-configure-shadowsocks-server.sh
#!/bin/bash
sudo apt-get -y update
sudo apt-get -y upgrade
sudo apt-get install -y python-gevent python-pip python-m2crypto supervisor
sudo pip install shadowsocks
sudo mkdir -p touch /etc/shadowsocks
You can’t perform that action at this time.