Johnny Chang hiimdoublej
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| FROM php:7.1.2-apache | |
| RUN docker-php-ext-install mysqli |
jbinfo
/ closespider.py
Last active
June 7, 2019 06:40
CloseSpider is a Scrapy extension that force spider to be closed after it reach a drop items limit
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # -*- coding: utf-8 -*- | |
| # MIT License (c) Lhassan Baazzi <baazzilhassan@gmail.com> | |
| from collections import defaultdict | |
| from twisted.internet import reactor | |
| from scrapy import signals | |
| class CloseSpider(object): | |
| def __init__(self, crawler): |
josephmosby
/ mongo-django.py
Created
March 25, 2015 15:03
Architecture for declaring a global MongoDB MongoClient in Django
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #project/project/__init__.py | |
| import project.settings | |
| import pymongo | |
| THE_MONGO_CLIENT = pymongo.MongoClient(settings.MONGO_HOST, settings.MONGO_PORT) | |
| #project/project/settings.py | |
| INSTALLED_APPS += 'utils' | |
| MONGO_HOST = 'localhost' #default |