- lxml - Pythonic binding for the C libraries libxml2 and libxslt.
- boto - Python interface to Amazon Web Services
- Django - Django is a high-level Python Web framework that encourages rapid development and clean, pragmatic design.
- Fabric - Library and command-line tool for streamlining the use of SSH for application deployment or systems administration task.
- PyMongo - Tools for working with MongoDB, and is the recommended way to work with MongoDB from Python.
- Celery - Task queue to distribute work across threads or machines.
- pytz - pytz brings the Olson tz database into Python. This library allows accurate and cross platform timezone calculations using Python 2.4 or higher.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
set -o errexit | |
source libs.sh | |
change_apt_source mirrors.aliyun.com | |
prepare | |
# prepare for python virtualenv | |
sudo apt-get install python-dev python-virtualenv | |
mkdir ~/.venv | |
virtualenv --system-site-packages ~/.venv/main |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
see the other files |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
testfile |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# -*- coding: utf-8 -*- | |
import requests | |
from bs4 import BeautifulSoup | |
import os | |
import os.path as op | |
def downtofile(url, dirname, filename): | |
dirpath = op.abspath(op.join(op.dirname(__file__), "downloads")) | |
if not op.exists(dirpath): | |
os.mkdir(dirpath) |