ResolveWang

import time

import requests

from weibospider.db.dao import WbDataOper


# 借助百度地图API实现地理位置的经纬度获取
class LocationHelper:
    # ak 是在 http://lbsyun.baidu.com/apiconsole/key 申请的

ResolveWang

class GoogleNewsSpider(RedisSpider, SpiderMixin):
    """spider for google news"""
    name = 'google_news'
    redis_key = 'search_engine:google_news'
    req_num = 10
    proxy_mode = 2

    custom_settings = {
        "DOWNLOAD_DELAY": 20,
        "CONCURRENT_REQUESTS_PER_DOMAIN": 1

ResolveWang

[unix_http_server]
file=/tmp/supervisor.sock   ; (the path to the socket file)

[supervisord]
logfile=/home/aaa/project/log/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB        ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10           ; (num of main logfile rotation backups;default 10)
loglevel=info                ; (log level;default info; others: debug,warn,trace)
pidfile=/tmp/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=false               ; (start in foreground if true;default false)

ResolveWang

def get_zero_timestamp(self):
    cur_timestamp = time.time()
    cur_time = time.localtime(cur_timestamp)
    zero_time_stamp = cur_timestamp - (
            cur_time.tm_sec + 60 * cur_time.tm_min + 3600 * cur_time.tm_hour)
    return zero_time_stamp

ResolveWang

# according to the blog:http://blog.csdn.net/Bone_ACE/article/details/55000101 

import socket
# from gevent import socket

_dnscache = {}

def _setDNSCache():
    """ DNS缓存 """

ResolveWang

Python Socket 编程详细介绍

Python 提供了两个基本的 socket 模块：

• Socket 它提供了标准的BSD Socket API。
• SocketServer 它提供了服务器重心，可以简化网络服务器的开发。

下面讲解下 Socket模块功能。

Socket 类型

ResolveWang

:: run it with administrator rights
netsh interface ipv6 isatap set router 202.115.39.98
netsh interface ipv6 isatap set state enabled

:: then assign your dns to the value of "2001:4860:4860::8888" or "2001:470:20::2"

ResolveWang

#!/bin/bash
# supervisord   This scripts turns supervisord on  
# chkconfig:    345 83 04  
# description:  supervisor is a process control utility.  It has a web based  
#               xmlrpc interface as well as a few other nifty features.  
#  

# source function library  
. /etc/rc.d/init.d/functions

ResolveWang

The Best of the Best Practices (BOBP) Guide for Python

A "Best of the Best Practices" (BOBP) guide to developing in Python.

In General

Values

• "Build tools for others that you want to be built for you." - Kenneth Reitz
• "Simplicity is alway better than functionality." - Pieter Hintjens

ResolveWang

# -*- coding: utf-8 -*-
# fabfile.py这个名字固定不能改变
from datetime import datetime
from fabric.api import env, local, cd, run, put, settings, lcd
from fabric.context_managers import prefix


def production():
    env.hosts = ["production@123.123.123.123:22"]
    env.key_filename = "/path/to/key_file"