Solve a deployment problem
Problem：psycopg2 not found
This afternoon I have met a problem for my website, it is not accessible due to an error of psycopg2, which is a module for python to connect with Postgresql database. I would like to document this to public for anyone in future for a referrence.
After the problem occurred, I blamed AWS, that everything was running OK and it just suddenly failed. The fact is that I didn't change anything in between, which is annoying. Or maybe one of the system upgrade suggested by AWS was the reason, I remembered that several packages was installed, but it seemed the problem had not really instantly occurred. But anyhow, the problem has to be solved, after waiting for serveral hours, I started to debug it. The problem states that:
ImproperlyConfigured: Error loading psycopg2 module: No module named psycopg2
which apprently means that
psycopg2 is not found by my web server.
Here is the settings of my system.
apache (httpd) Django 1.6 python 2.7 AWS EC2 with centOS 6.4 mod_wsgi 3.5 virtualenv 1.9
solution：recompiling mod_wsgi and reinstalling httpd
I google around and found something similar as mine. So I checked my path of virtualenv and everything, all seem in correct ways. Finally I decided to recompile mod_wsgi, from the hint projected by this post. Actually, when I first deploy my website, I have met similar problem, though not exactly as this one, and I have documented this post for later use. It was a surprise to return to this post. On that document, I even marked that section where this post is recorded as a "Caution" part. So I decided to follow the solution in that post, re-compliling mod_wsgi.
As I proceed, just found that mod_wsgi has a new version availible, and it mentions that the previous version of 3.4 has security issues, so why not just update it to the newest. So I download the 3.5 version, compile it under my virtualenv, so that it is compatible with python27 rather than python2.6.9 which is the defaul python of centOS 6.4 of EC2.
So is it solved? No. Not yet.
So I tried twice with system installed python2.7 but without virtualenv, and only virtualenv, but both failed to solve the problem. So I decided to re-install httpd as well.
sudo yum remove httpd
and httpd with two dependencies also were removed， one is httpd-devel, another is mod_wsgi 3.2 .
So I reinstall httpd and finally the problem solved. From this experience, I re-discovered that to document in detail is how much of importance. So I have to record how I deploy Django on AWS EC2.
Deploy Django on AWS EC2
I was much lighted by this post, this tutorial is based on it, while I add more details and explanations here as well.
I chose centOS AMI, which is recommended by AWS. Another main choice is Ubuntu.
install python 27
Note that the default CentOS installation of Python is 2.6.9, we better upgrade it to 2.7. Before this, you might optionally update all packages first, which is
sudo yum update, but I just skip it.
sudo yum install gcc python27 python27-devel
gcc is for compiling a lot modules, so better install it as soon as possible.
python27-devel is centOS compatible packages for compiling python related packages.
in case not for yum install on AWS EC2:
** for aliyun **
yum install python-devel.x86_64 openssl-devel
** for compiling python2.7 from source **
./configure —enable-shared (for postgresql to use) make && make install
for problem related with
shared lib referrence.
run commands as root user
Note that a series of commands need to be run with sudo previledge, so better do it once,
sudo su - root
now we can install software without prefix
yum install httpd httpd-devel
curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.9.1.tar.gz tar -xvf virtualenv-1.9.1.tar.gz python27 virtualenv-1.9.1/virtualenv.py ~/venv-27 source ~/venv-27/bin/activate
curl and wget is used interchangeably.
for https connection failure, add this option:
for zlib problems of virtualenv, can first install
yum install zlib-devel
then reinstall python27
venv-27 is a folder that holds the virtual env packages, can be customized as you wish for its location and name
install this if not yet, because it is a dependency of mod_wsgi
yum install httpd httpd-devel
wget https://github.com/GrahamDumpleton/mod_wsgi/archive/3.5.tar.gz tar -xvf 3.5.tar.gz ./configure (note that right now should be in virtualenv) make && make install (note that now should be with root user privilege)
- virtualenv can help compilation compatible with python27
either add a file to
echo 'LoadModule wsgi_module modules/mod_wsgi.so' > /etc/httpd/conf.d/wsgi.conf
or just add another line to
LoadModule wsgi_module modules/mod_wsgi.so
and change the user which run httpd to ec2-user
User ec2-user Group ec2-user
before installing pip, you might need this package, if not installed by default, for downloading from internet sources.
yum install openssl-devel easy_install pip
for Postgresql users, install psycopg2
sudo yum install postgresql-devel pip install psycopg2
install Django and other packages
(venv-27)$ pip install Django==1.6
or from a file
pip install -r requirements-python27.txt
for Mac user
pip sometimes not work due to xcode 5.1
ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future pip install -r requirements-python27.txt
install the app, and the database is at
setup virtual host for httpd
add these lines in file
httpd.conf at the bottom part, you need
sudo vi /etc/httpd/conf/httpd.conf
WSGIPythonPath /home/ec2-user/weblinwo:/home/ec2-user/venv-27/lib/python2.7/site-packages <VirtualHost *:80> # ServerAdmin firstname.lastname@example.org ServerAdmin email@example.com # DocumentRoot /www/docs/dummy-host.example.com DocumentRoot /home/ec2-user/web # ServerName dummy-host.example.com ServerName ec2-123-123-123-123.xxxxxx.compute.amazonaws.com Timeout 60 WSGIScriptAlias / /home/ec2-user/web/youwebapp/wsgi.py <Directory /home/ec2-user/web/youwebapp> <Files wsgi.py> Order allow,deny Allow from all </Files> </Directory> AliasMatch ^/([^/]*\.css) /home/ec2-user/static/css/$1 #Alias /media/ /path/to/mysite.com/media/ Alias /static/ /home/ec2-user/static/ Alias /favicon.ico /path/to/favicon.ico <Directory /home/ec2-user/static> Allow from all </Directory> <Directory /> AllowOverride None Options Indexes </Directory> <Directory /home/ec2-user/web/.git/> Deny From All </Directory> ErrorLog /home/ec2-user/httpdlogs/error_log CustomLog /home/ec2-user/httpdlogs/access_log combined </VirtualHost>
start httpd server
sudo service httpd start
sudo cat /etc/httpd/run/httpd.pid
ps -A ps aux
after installed psql
database create user
createdb -T template0 db_mba_withyou_pg
pg_dump —username=Username —dbname=dbname —file=backupfile
consider use RDS
so it will be easily scaled
AWS Security setup
from outside to connect to server，ask for enter server
icmp setup finished, then you can ping from local machine to public dns of ec2
from server to transfer data to outside，or from server to connect to other sources icmp setup finished, then you can ping from ec2 to public websites
RDS, inbound settings，open
5432 port of custom tcp
ec2, outbound settings，also should open this port