This is a condensed step-by-step guide on how to install Marketplace with all its co-servers (redis, ES, Rabittmq, MySQL etc)
More on Marketplace/Zamboni: http://zamboni.readthedocs.org/
- Grab the Ubuntu Server 12.04.1 image at http://www.ubuntu.com/start-download?distro=server&bits=64&release=lts
- Make sure you have Virtualbox installed on your system
- Create an image with the Ubuntu iso
- add a new network adapter on that image. Host-Only against vboxnet0. vboxnet's ipaddr should be 33.33.33.1
- Run it !
- when you install Ubuntu, use the default settings and create a user marketplace, password marketplace (or wathever you want - just adapt the tutorial)
once you are logged in your fresh ubuntu, run the following:
sudo apt-get install python-software-properties sudo add-apt-repository ppa:fkrull/deadsnakes sudo apt-get update sudo aptitude install python python2.6 python2.6-dev python-virtualenv libxml2-dev libxslt1-dev libmysqlclient-dev libmemcached-dev libssl-dev swig git mysql-server mysql-client subversion g++ make nginx rabbitmq-server redis-server openjdk-7-jre libjpeg-dev libfreetype6 libfreetype6-dev zlib1g-dev ffmpeg
Ubuntu will ask you for a root password for MySQL, use root (or wathever you want - just adapt the tutorial)
This going to last for ages if you live in the country like me.
Last but not least, edit the /etc/network/interfaces file in your fresh Ubuntu, so we define a static ip address to the second adapter add this:
auth eth1
iface eth1 inet static
address 33.33.33.24
netmask 255.255.255.0
gateway 33.33.33.1
Restart the network and make sure eth1 show up with the right IP:
sudo /etc/init.d/networking restart
ifconfig
Try to ping the VBOx from your host:
ping 33.33.33.24
You should be ablt to reach it, and the VBox should be able to reach the net through the host.
Lessc is needed by Marketplace to create all the css. Ubuntu has a node-less package but I had issues with it, so I did a manual installation of node, nmp and lessc:
cd /tmp
git clone https://github.com/joyent/node.git
cd node && ./configure && make
sudo make install
cd ~
sudo curl -s https://npmjs.org/install.sh | sh
sudo npm install less -g
sudo npm install uglify-js -g
Manual install here, like lessc :/ -- taken from https://gist.github.com/2026107 :
cd /tmp
wget https://github.com/downloads/elasticsearch/elasticsearch/elasticsearch-0.19.9.tar.gz -O elasticsearch.tar.gz
tar -xf elasticsearch.tar.gz
rm elasticsearch.tar.gz
sudo mv elasticsearch-* elasticsearch
sudo mv elasticsearch /usr/local/share
curl -L http://github.com/elasticsearch/elasticsearch-servicewrapper/tarball/master | tar -xz
mv *servicewrapper*/service /usr/local/share/elasticsearch/bin/
rm -Rf *servicewrapper*
sudo /usr/local/share/elasticsearch/bin/service/elasticsearch install
sudo ln -s `readlink -f /usr/local/share/elasticsearch/bin/service/elasticsearch` /usr/local/bin/rcelasticsearch
sudo service elasticsearch start
Next, edit /usr/local/share/elasticsearch/config/elasticsearch.yml and add:
cluster:
name: wooyeah
index:
analysis:
analyzer:
standardPlusWordDelimiter:
type: custom
tokenizer: standard
filter: [standard, wordDelim, lowercase, stop, dict]
filter:
wordDelim:
type: word_delimiter
preserve_original: true
dict:
type: dictionary_decompounder
word_list_path: wordlist.txt
Then go into the config dir and download the wordlist:
cd /usr/local/share/elasticsearch/config
wget https://bug674810.bugzilla.mozilla.org/attachment.cgi?id=549013 -O wordlist.txt
Last, restart it:
sudo /etc/init.d/elasticsearch restart.
Marketplace uses RabbitMQ through Celery to defer some tasks. It was installed previously. Just run it and add a zamboni user:
sudo rabbitmq-server -detached
sudo rabbitmqctl add_user zamboni zamboni
sudo rabbitmqctl add_vhost zamboni
sudo rabbitmqctl set_permissions -p zamboni zamboni ".*" ".*" ".*"
MySQL should be already running. Let's create a marketplace user and DB:
mysqladmin -uroot -proot
$(mysql shell) CREATE USER 'marketplace'@'localhost' IDENTIFIED BY 'marketplace';
$(mysql shell) CREATE DATABASE marketplace;
$(mysql shell) GRANT ALL PRIVILEGES ON marketplace.* to marketplace@localhost ;
$(mysql shell) exit
Let's install Zamboni now, which is the application that powers Marketplace:
cd ~
git clone --recursive git://github.com/mozilla/zamboni.git
cd zamboni
svn co http://svn.mozilla.org/addons/trunk/site/app/locale locale
curl -s https://raw.github.com/brainsik/virtualenv-burrito/master/virtualenv-burrito.sh | $SHELL
source ~/.venvburrito/startup.sh
mkvirtualenv marketplace -p python2.6
What I did here is cloned Zamboni and all its submodules, and created a virtualenv for it. For now on if you run Python, it will be the virtualenv one. If you exit it, you can just run:
workon marketplace
Now set up the pip download cache by creating ~/.pip/pip.conf with :
[global]
download-cache = /home/marketplace/.pip/downloads
That will reuse already downloaded files everytime you run Pip. This will speed things up for the next runs.
Then:
pip install -r requirements/dev.txt
Now let's configure the app settings:
cd ~/zamboni
cp docs/settings/settings_local.dev.py settings_local_mkt.py
ln -s settings_local_mkt.py settings_local.py
Now let's edit the file. You want to add those:
LESS_LIVE_REFRESH = True
LESS_PREPROCESS = True
SERVE_TMP_PATH = True
RUN_ES_TESTS = True
LESS_BIN = 'lessc'
UGLIFY_BIN = 'uglifyjs'
SITE_URL_OVERRIDE = SITE_URL = STATIC_URL = 'http://addons.mozilla.dev'
Then change the Databse configuration so it looks like this:
DATABASES = {
'default': {
'NAME': 'marketplace',
'ENGINE': 'django.db.backends.mysql',
'USER': 'marketplace',
'PASSWORD': 'marketplace',
'OPTIONS': {'init_command': 'SET storage_engine=InnoDB'},
'TEST_CHARSET': 'utf8',
'TEST_COLLATION': 'utf8_general_ci',
},
}
Here's an example of a full settings file: https://gist.github.com/3818180
We want to do 3 things: - fill the My SQL DB with the initial structure - compress the assets - get Elastic Search ready
For the DB SQL, run this command:
python manage.py --settings=settings_local_mkt install_landfill
schematic migrations
I had several issues here. First I had to run the migration #260 manually. Second I had issues in django-mozilla-product-details. For the latter, have a look at https://github.com/fwenzel/django-mozilla-product-details/issues/27 . Applying that fix worked for me.
The next step is to compress the assets:
python manage.py --settings=settings_local_mkt compress_assets
That runs lessc and uglifyjs.
Last, let's prepare the Elastic Search DB:
python manage.py cron reindex_addons
We're almost done !
Since Marketplace uses BrowserId, we need to trick the Persona server. Add this in your /etc/hosts file (not in the Virtualbox guest!)
33.33.33.24 addons.mozilla.dev
Last, let's configure Nginx within the VM so it binds the port 80 and proxies all the requests to Django. I just changed the file at /etc/nginx/site-enabled/default, so the location / section looks like this:
location / {
proxy_pass http://0.0.0.0:8000:
proxy_set_header Host $host;
}
restart nginx:
sudo /etc/init.d/nginx restart
And start Django:
cd ~/zamboni
python manage.py runserver --settings=settings_local_mkt 0.0.0.0:8000
Then visit http://addons.mozilla.dev and try to log in.
Install nose-progressive:
pip install nose-progressive
Then run the tests :
python manage.py test --noinput -s --logging-clear-handlers --with-progressive --settings=settings_local_mkt mkt