Skip to content

Instantly share code, notes, and snippets.

View wppurking's full-sized avatar

wppurking wppurking

62 followers · 7 following
View GitHub Profile
@wppurking
wppurking / output.sh
Last active April 9, 2018 05:37
nsqd-ruby debugger
wyatt@mac:nsq/src $ ruby producer.rb
SizeQueue: 2495
SizeQueue: 4947
SizeQueue: 7446
SizeQueue: 9999
SizeQueue: 9891
SizeQueue: 10000
SizeQueue: 10000
SizeQueue: 10000
SizeQueue: 9916
@wppurking
wppurking / Procfile
Created April 9, 2018 04:59
nsq-ruby producer send message will lose
# statsd & graphite
sg: docker run --name graphite --rm -p 80:80 -p 2003-2004:2003-2004 -p 2023-2024:2023-2024 -p 8125:8125/udp -p 8126:8126 hopsoft/graphite-statsd
# nsqdlookupd http:4161, tcp:4160
nsqdlookupd: nsqlookupd
# nsqd http:4151, https: 4152, tcp: 4150
nsqd: nsqd -lookupd-tcp-address=127.0.0.1:4160 -mem-queue-size=0 -broadcast-address=127.0.0.1 -statsd-address="127.0.0.1:8125" -statsd-interval="10s"
nsqd2: nsqd -lookupd-tcp-address=127.0.0.1:4160 -mem-queue-size=0 -tcp-address=0.0.0.0:4250 -http-address=0.0.0.0:4251 -https-address=0.0.0.0:4252 -data-path=./nsqd2 -broadcast-address=127.0.0.1 -statsd-address="127.0.0.1:8125" -statsd-interval="10s"
@wppurking
wppurking / bench.rb
Last active January 30, 2018 07:54
roar vs ams vs rabl vs jbuilder Raw. (json_gem, oj)
require 'bundler'
require 'active_model_serializers'
require 'roar/decorator'
require 'roar/json'
require 'rabl'
require 'jbuilder'
require 'benchmark'
require 'ffaker'
require 'multi_json'
require 'oj'
@wppurking
wppurking / login_twitter.rb
Created December 17, 2017 02:16
使用 capybara 以及 selenium_chrome 登陆 Twitter 后台
require 'capybara'
require 'selenium-webdriver'
require 'thread/pool'
pool = Thread.pool(3)
Capybara.threadsafe = true
Capybara.default_driver = :selenium_chrome
#Capybara.default_driver = :selenium_chrome_headless
@wppurking
wppurking / gist:dd33338f601743dca2db300cbbd48de4
Created October 13, 2017 10:05
baf5 url="https://www.amazon.com/product-reviews/B0018OPL7O/ref=cm_cr_arp_d_viewopt_rvwer?ie=UTF8&reviewerType=all_reviews&pageSize=50&sortBy=recent&filterByStar=all_stars"
fatal error: concurrent map read and map write
goroutine 717 [running]:
runtime.throw(0xb22bc8, 0x21)
/usr/local/Cellar/go/1.9/libexec/src/runtime/panic.go:605 +0x95 fp=0xc420982608 sp=0xc4209825e8 pc=0x42a375
runtime.mapaccess2(0xa47860, 0xc420454390, 0xc420df3410, 0xa02be0, 0x4dfcc4)
/usr/local/Cellar/go/1.9/libexec/src/runtime/hashmap.go:413 +0x24e fp=0xc420982650 sp=0xc420982608 pc=0x40847e
reflect.mapaccess(0xa47860, 0xc420454390, 0xc420df3410, 0xc420454390)
/usr/local/Cellar/go/1.9/libexec/src/runtime/hashmap.go:1218 +0x3f fp=0xc420982688 sp=0xc420982650 pc=0x40ad9f
@wppurking
wppurking / gist:08f1197d7bdb7f1bee36e2e50a1c7f62
Created October 13, 2017 08:09
```
DEBU[2017-10-13 08:06:41] discover_review_task.poster 任务结束 duration=0.394015634 jid=28b25547b9b04da8ef051d39 job=discover_review_task.poster slow=false source="/jobs/context.go[38]: jobs.(*Context).logDuration"
fatal error: concurrent map writes
goroutine 1096 [running]:
runtime.throw(0xb1aad8, 0x15)
/usr/local/Cellar/go/1.9/libexec/src/runtime/panic.go:605 +0x95 fp=0xc420f46ac8 sp=0xc420f46aa8 pc=0x42a375
runtime.mapassign_faststr(0xa47860, 0xc42178e120, 0xb11b22, 0x3, 0x3)
/usr/local/Cellar/go/1.9/libexec/src/runtime/hashmap_fast.go:607 +0x4f5 fp=0xc420f46b48 sp=0xc420f46ac8 pc=0x40cc65
ear-monitor/models.(*taskRouter).Rule(0xc4201bd840, 0xc4222da480, 0xc420f46d00, 0xc420f46d08, 0xc421ee2580)
@wppurking
wppurking / ES.json
Created March 14, 2017 06:22
[
{
"tid": "0068fbe3-bd16-43d5-bfe1-cd0a940fd013",
"url": "https://www.amazon.es/product-reviews/B01G1D7ID6/ref=cm_cr_arp_d_viewopt_srt?ie=UTF8&filterByStar=all_stars&pageNumber=1&pageSize=50&sortBy=recent",
"callback": "http://44c74ebf.ngrok.io/api/v1/echo",
"rules": [
{
"scope_rule": "//*[@id='cm_cr-review_list']/div[contains(@class,'review')]",
"kv_rule": [
{
@wppurking
wppurking / DE.json
Created March 14, 2017 06:21
[
{
"tid": "0162d346-dae0-40a3-b0c3-8aa1fa312870",
"url": "https://www.amazon.de/product-reviews/B01I94NJ1U/ref=cm_cr_arp_d_viewopt_srt?ie=UTF8&filterByStar=all_stars&pageNumber=1&pageSize=50&sortBy=recent",
"callback": "http://44c74ebf.ngrok.io/api/v1/echo",
"rules": [
{
"scope_rule": "//*[@id='cm_cr-review_list']/div[contains(@class,'review')]",
"kv_rule": [
{
@wppurking
wppurking / CA.json
Last active March 14, 2017 06:20
[
{
"tid": "04e9d161-5139-406c-b9ff-e2f96e37a3c5",
"url": "https://www.amazon.ca/product-reviews/B015O9O5D4/ref=cm_cr_arp_d_viewopt_srt?ie=UTF8&filterByStar=all_stars&pageNumber=1&pageSize=50&sortBy=recent",
"callback": "http://44c74ebf.ngrok.io/api/v1/echo",
"rules": [
{
"scope_rule": "//*[@id='cm_cr-review_list']/div[contains(@class,'review')]",
"kv_rule": [
{
@wppurking
wppurking / mail.eml
Created February 20, 2017 11:34
X-Alimail-AntiSpam:AC=SPAM;BC=0.9200199|0.8529158;BR=01201311R951b1;FP=0|0|0|0|0|-1|-1|-1;HT=e02c03308;MF=rte+ne-null-b1cb1a0678232epoxisg1c7dr@sellernotifications.amazon.com;NM=1;PH=DM;RN=1;RT=1;SC=102.3;SR=6;TI=MTA_---.7fVm9dC_1487574683;
Received: from a1-235.smtp-out.eu-west-1.amazonses.com(mailfrom:RTE+NE-null-b1cb1A0678232EPOXISG1C7DR@sellernotifications.amazon.com ip:54.240.1.235)
by mx1.aliyun-inc.com(10.147.42.34);
Mon, 20 Feb 2017 15:11:26 +0800
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple;
s=yqjhwsipgo3kjycfsffiutnhfur6rrw6; d=amazon.es; t=1487574670;
i=@marketplace.amazon.es;
h=Date:From:Reply-To:To:Message-ID:Subject:MIME-Version:Content-Type;
bh=RukVGddkhEQujy9d3mm8dHWExq09y+CqL32FcQtaKC8=;
b=GteVz1AZ8NRMBDi0hqoSo4oW5keLPOOYP9Xby5OSR+sfpjRJfFGk1sUOBJSNwpKB