Skip to content

Instantly share code, notes, and snippets.

@jlyon
Last active December 24, 2015 01:29
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jlyon/6724403 to your computer and use it in GitHub Desktop.
Save jlyon/6724403 to your computer and use it in GitHub Desktop.
Getting started with Vufind 2.1 and demo MARC records on Ubuntu 13.04

Getting started with Vufind

Installing

See http://vufind.org/wiki/vufind2:installation_ubuntu. We're just doing the fastest installation with the .deb package. Make sure that your security group (if using AWS) is set to allow ports 80 and 8080.

Install Vufind and Dependencies

Create an Ubuntu 13.04 instance in Amazon AWS and login as root (sudo su).

# Dependencies
sudo apt-get update
sudo apt-get dist-upgrade
sudo shutdown -r now

sudo apt-get -y install git
sudo apt-get -y install apache2
sudo a2enmod rewrite
sudo /etc/init.d/apache2 force-reload
sudo apt-get -y install mysql-server
sudo apt-get -y install php5 php5-dev php-pear php5-ldap php5-mcrypt php5-mysql php5-xsl php5-intl php5-gd php5-json
sudo apt-get -y install default-jdk

# We needed sybase to connect to Horizon ILS
sudo apt-get -y install php5-sybase

# Ubuntu 13.04 requires a little extra help installing mcrypt
# From: http://ubuntuforums.org/showthread.php?t=2184310
cd /etc/php5/mods-available
sudo ln -s ../conf.d/mcrypt.ini
sudo php5enmod mcrypt

# Manually install Vufind from git repo
cd /usr/local
git clone --branch release-2.1 https://github.com/vufind-org/vufind.git
cd vufind
php install.php

# Setup system variables
sudo sh -c 'echo export VUFIND_HOME=\"/usr/local/vufind\"  >> /etc/profile'
sudo sh -c 'echo export VUFIND_LOCAL_DIR=\"/usr/local/vufind/local\"  >> /etc/profile'
sudo sh -c 'echo export JAVA_HOME=\"/usr/lib/jvm/default-java\" >> /etc/profile'
source /etc/profile

# Permissions
sudo chown -R www-data:www-data /usr/local/vufind
sudo chown -R www-data:www-data /usr/local/vufind/local/cache
sudo chown -R www-data:www-data /usr/local/vufind/local/config

# Setting up Vufind commandline tools
mkdir /usr/local/vufind/local/cache/cli
chmod 777 /usr/local/vufind/local/cache/cli

# Setup Apache
ln -s /usr/local/vufind/local/httpd-vufind.conf /etc/apache2/conf-enabled/vufind.conf
sudo service apache2 restart

Note: I had to slightly modify /usr/local/vufind/local/httpd-vufind.conf:

  • Comment out Order allow,deny and allow from all lines
  • Below the 2 commented out lines, add Require all granted. It seems like there were some apache changes in 13.04. Restart apache again.

Install SOLR

I decided to run SOLR locally for testing and development. You might want to put this on a separate server in production. From: http://vufind.org/wiki/vufind2:remote_solr_back-end.

Install SOLR:

sudo apt-get install tomcat7
cd ~/
wget http://archive.apache.org/dist/lucene/solr/4.2.0/solr-4.2.0.tgz
tar xvzf solr-4.2.0.tgz
sudo cp solr-4.2.0/dist/solr-4.2.0.war /var/lib/tomcat7/webapps/solr.war
sudo mkdir -p /usr/local/share/solr
sudo mkdir -p /etc/solr/cores
sudo mkdir -p /usr/local/lib/solr
sudo mkdir -p /var/lib/solr/data
sudo chown tomcat7:tomcat7 /var/lib/solr/data
sudo ln -s /etc/solr/solr.xml /usr/local/share/solr/solr.xml
sudo ln -s /etc/solr/cores /usr/local/share/solr/cores
sudo ln -s /var/lib/solr/data /usr/local/share/solr/data
sudo ln -s /usr/local/lib/solr /usr/local/share/solr/lib

Configure SOLR with vufind config files

cd /usr/local/vufind
cp solr/solr.xml /etc/solr/solr.xml
mkdir /etc/solr/cores
cp -r solr/authority/ solr/biblio/ solr/reserves/ solr/stats/ /etc/solr/cores
sudo chown tomcat7:tomcat7 /etc/solr/cores
sudo cp ~/solr-4.2.0/contrib/analysis-extras/lib/* solr-4.2.0/contrib/analysis-extras/lucene-libs/* /usr/local/lib/solr
sudo cp /usr/local/vufind/solr/lib/browse-handler.jar /usr/local/lib/solr

Modify /etc/solr/solr.xml to:

<?xml version="1.0" encoding="UTF-8" ?>
<solr persistent="false" sharedLib="lib">
  <cores adminPath="/admin/multicore">
    <core name="biblio" instanceDir="cores/biblio" dataDir="/var/lib/solr/data/biblio" />
    <core name="authority" instanceDir="cores/authority" dataDir="/var/lib/solr/data/authority" />
    <core name="stats" instanceDir="cores/stats" dataDir="/var/lib/solr/data/stats" />
    <core name="reserves" instanceDir="cores/reserves" dataDir="/var/lib/solr/data/reserves" />
  </cores>
</solr>

Add -Dsolr.solr.home=/usr/local/share/solr to JAVA_OPTS in /etc/default/tomcat7 (you might also want to increase Java's heap size).

Restart solr (via tomcat)

sudo service tomcat7 restart

You can access the SOLR Admin Console at http://localhost:8080/solr/.

Running Vufind for the first Time and Configuring

To start Vufind:

cd /usr/local/vufind/
./vufind.sh start

You can access the site at http://localhost/vufind/Install/Home. Click on Fix for each of the failed items. Some settings:

  • Create new database: u:p I used was vufind:vufind.
  • @todo: ILS: Select Horizon, need to edit /usr/local/vufind/local/config/vufind/Horizon.ini
Setting up upstart

Upstart makes it really easy to start/stop/restart vufind without having to enter the full path.

To install (more info):

cd nano /etc/init
wget https://gist.github.com/jlyon/6724403/raw/96528c39113a78f9f2863b7f43b88b75a60c0429/vufind.conf

To use upstart:

start vufind
stop vufind
restart vufind

@todo: is this necessary? I recommend automatically starting vufind on every boot. To do this, edit your crontab: crontab -e and add:

@reboot start vufind

Load in MARC data

Following along http://vufind.org/wiki/importing_records.

We are going to use demo MARC data from: http://www.lib.umich.edu/open-access-bibliographic-records via http://wiki.koha-community.org/wiki/Free_sample_MARC_data. (Obviously if you have real MARC records, use them in place of these).

cd ~/
wget http://www.lib.umich.edu/files/umich_bib.marc.gz
gunzip umich_bib.marc.gz

Before we can load in data, we need to edit /usr/local/vufind/import/import.properties:

  • Change
Import the data with solrmarc.

Before doing this, increase the system memory (change the AWS EC2 instance type).

cd /usr/local/vufind/
./import-marc.sh ~/umich_bib.marc

Some stats when I ran this on an AWS EC2 medium instance:

 INFO [main] (MarcImporter.java:627) - Finished indexing in 62:18.00
 INFO [main] (MarcImporter.java:636) - Indexed 685601 at a rate of about 183.0 per sec
Delete data from solr index

See the attached vufind_solr_rebuild.sh file to delete all SOLR entries and reload MARC records.

#!upstart
# This is the vufind upstart script
description "vufind"
start on started mountall
stop on shutdown
# Automatically Respawn:
respawn
respawn limit 99 5
exec bash /usr/local/vufind/vufind.sh
# Delete existing solr entries
curl http://localhost:8080/solr/authority/update?stream.body=%3Cdelete%3E%3Cquery%3E*:*%3C/query%3E%3C/delete%3E&commit=true
curl http://localhost:8080/solr/biblio/update?stream.body=%3Cdelete%3E%3Cquery%3E*:*%3C/query%3E%3C/delete%3E&commit=true
curl http://localhost:8080/solr/reserves/update?stream.body=%3Cdelete%3E%3Cquery%3E*:*%3C/query%3E%3C/delete%3E&commit=true
curl http://localhost:8080/solr/stats/update?stream.body=%3Cdelete%3E%3Cquery%3E*:*%3C/query%3E%3C/delete%3E&commit=true
# Load in new data
bash /usr/local/vufind/import-marc.sh ~/vufindfully8.mrc
# Restart Vufind
bash /usr/local/vufind/vufind.sh restart
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment