Skip to content

Instantly share code, notes, and snippets.

@petemcw
Last active January 18, 2020 03:45
Show Gist options
  • Star 5 You must be signed in to star a gist
  • Fork 5 You must be signed in to fork a gist
  • Save petemcw/5341060 to your computer and use it in GitHub Desktop.
Save petemcw/5341060 to your computer and use it in GitHub Desktop.
Basic configuration steps for setting up Solr 3.x on CentOS/RHEL 6.4 -- specifically for use with Magento Enterprise

Installing Solr on CentOS/RHEL 6

From the Apache Software Foundation, "Solr is an open source enterprise search server based on the Lucene Java search library, with XML/HTTP and JSON, Ruby, and Python APIs, hit highlighting, faceted search, caching, replication, and a web administration interface." Magento has had built-in support for Solr since Enterprise verion 1.8.

The following instructions show how to complete a basic installation and configuration of Apache Solr for use with Magento Enterprise on CentOS/RHEL 6. These instructions the Java JDK with Tomcat rather than jetty.

Note: Solr should be on a dedicated server if possible, especially for large installations, as it can be resource intensive

Install Java JDK

Remove any old versions of Java and install the latest 1.6 version from OpenJDK.

$› sudo yum remove java
$› sudo yum install -y java-1.6.0-openjdk java-1.6.0-openjdk-devel

Install Apache Tomcat

Install Tomcat and slightly configure the directory structure from default; this will keep things all together and tidy.

$› sudo yum install -y tomcat6 tomcat6-webapps tomcat6-admin-webapps

Make sure Tomcat is started when the server boots up.

$› chkconfig tomcat6 on

Configure Firewall

Ensure your firewall is configured to open TCP port 8080 to the desired hosts, as this is the default port Tomcat & Solr communicate on. If using iptables, below is an example configuration for creating a chain to handle Solr access:

$› sudo -s
#› iptables -N SOLR
#› iptables -A SOLR --src 192.168.1.100 -j ACCEPT
#› iptables -A SOLR -j DROP
#› iptables -I INPUT -m tcp -p tcp --dport 8080 -j SOLR
#› iptables-save > /etc/sysconfig/iptables
#› logout

Install Apache Solr

There are a few ways to get Solr installed, but here we will simply grab the precompiled tarball (rather than source). At the time of this writing the stable 3.x version of Solr is 3.6.2. Uncompress the file and add it to our Tomcat installation.

$› sudo wget https://archive.apache.org/dist/lucene/solr/3.6.2/apache-solr-3.6.2.tgz
$› sudo tar -zxvf apache-solr-3.6.2.tgz
$› sudo cp apache-solr-3.6.2/dist/apache-solr-*.war /var/lib/tomcat6/webapps/solr.war
$› sudo mkdir -p /var/lib/tomcat6/solr

Configure Tomcat to run Solr

After everything is installed, Tomcat must be configured to run Solr as a "webapp". Add the following configuration to get up and running.

Create a solr.xml context fragment for Catalina:

$› sudo vim /etc/tomcat6/Catalina/localhost/solr.xml

The contents of solr.xml should be (the commented out section adds IP address restrictions):

<Context docBase="/var/lib/tomcat6/webapps/solr.war" debug="0" privileged="true" allowLinking="true" crossContext="true">
    <Environment name="solr/home" type="java.lang.String" value="/var/lib/tomcat6/solr" override="true" />
    <!-- <Valve className="org.apache.catalina.valves.RemoteAddrValve" allow="127.0.0.1,192.168.1.100" /> -->
</Context>

Setup Multiple Solr Cores

Solr can run multiple "instances" with-in a single server installation. We'll create two different cores, one for each server environment.

$› sudo -s
#› cat > /var/lib/tomcat6/solr/solr.xml << EOF
<?xml version="1.0" encoding="UTF-8" ?>
<solr persistent="true" sharedLib="lib">
    <cores adminPath="/admin/cores">
      <core name="staging" instanceDir="staging" config="solrconfig.xml" schema="schema.xml" />
      <core name="production" instanceDir="production" config="solrconfig.xml" schema="schema.xml" />
    </cores>
</solr>
EOF
#› logout

Add Magento Solr Configuration

The following script easily adds the custom Magento configuration that ships with Enterprise. Take care to tweak the script to your specific installation needs.

INSTALL_DIR="/var/lib/tomcat6/solr"
touch $INSTALL_DIR/solr.xml
CORES=( "staging" "production" )
for CORE in "${CORES[@]}"; do
    mkdir -p $INSTALL_DIR/$CORE/conf $INSTALL_DIR/$CORE/data
    cp -par /usr/src/solr/apache-solr-3.6.2/example/solr/conf/* $INSTALL_DIR/$CORE/conf
    cp -par /var/www/magento/lib/Apache/Solr/Conf/* $INSTALL_DIR/$CORE/conf
done

Clean-up

For Tomcat to run successfully and Solr to create its indexes, make sure correct permissions are set.

$› sudo chown -R tomcat:tomcat /var/lib/tomcat6/ /etc/tomcat6/Catalina/localhost/solr.xml
$› sudo /etc/init.d/tomcat6 start

Tomcat Roles (Optional)

Tomcat can manage roles and permissions to individual webapps. If so desired, create a user for use with the above Solr configuration.

$› sudo vim /etc/tomcat6/tomcat-users.xml

Add the below user and roles.

<role rolename="manager"/>
<role rolename="admin"/>
<user username="tomcat" password="secret" roles="manager,admin"/>

You must tell the solr webapp to use the above ACL configuration.

$› sudo vim /var/lib/tomcat6/webapps/solr/WEB-INF/web.xml

Add in the security constraints.

<security-constraint>
    <web-resource-collection>
        <web-resource-name>Solr Lockdown</web-resource-name>
        <url-pattern>/</url-pattern>
    </web-resource-collection>
    <auth-constraint>
        <role-name>manager</role-name>
        <role-name>admin</role-name>
    </auth-constraint>
</security-constraint>
<login-config>
    <auth-method>BASIC</auth-method>
    <realm-name>Solr Administration</realm-name>
</login-config>

Make sure Tomcat is restarted to make the changes take effect.

$› sudo /etc/init.d/tomcat6 restart

Magento Solr Configuration

If using the current stable 3.x line of Solr, there are a few changes that can be made to Magento's solrconfig.xml.

$› vim /var/www/magento/lib/Apache/Solr/conf/solrconfig.xml

Make the following adjustments:

  • After the opening <config> block, add <luceneMatchVersion>LUCENE_30</luceneMatchVersion>
  • Merge the tags from <indexDefaults> and <mainIndex> into a single <indexConfig> block

Default Search Behavior (optional)

Magento's included configuration defaults to using "OR" for multiple search terms. It can be changed to an "AND|OR" type of search. For example, "black shirt" could return both "black AND shirt" and "black OR shirt" results instead of just the latter.

The change must be made in schema.xml.

$› vim /var/www/magento/lib/Apache/Solr/conf/schema.xml

If you want to change the operator behavior, replace:

<solrQueryParser defaultOperator="OR"/>

with:

<solrQueryParser defaultOperator="AND|OR"/>

Make sure Tomcat is restarted to make the changes take effect.

$› sudo /etc/init.d/tomcat6 restart

Magento Patch

You must apply this patch if using Solr version 3.6.1 or greater. Without the patch you will get errors that manifest in the following when searching or browsing layered navigation:

Notice: Undefined index: faceted_data ...

You can apply the included patch EE_1.12.0.2_v1.patch (from Magento Core team) by running the following in your Magento root directory.

$› patch -p0 < EE_1.12.0.2_v1.patch

Sources

Thanks for the help!

diff --git app/code/core/Enterprise/Search/Model/Adapter/Abstract.php app/code/core/Enterprise/Search/Model/Adapter/Abstract.php
index dbaa51c..3e2afd2 100644
--- app/code/core/Enterprise/Search/Model/Adapter/Abstract.php
+++ app/code/core/Enterprise/Search/Model/Adapter/Abstract.php
@@ -89,7 +89,7 @@ abstract class Enterprise_Search_Model_Adapter_Abstract
*/
protected $_defaultQueryParams = array(
'offset' => 0,
- 'limit' => 100,
+ 'limit' => Enterprise_Search_Model_Adapter_Solr_Abstract::DEFAULT_ROWS_LIMIT,
'sort_by' => array(array('score' => 'desc')),
'store_id' => null,
'locale_code' => null,
@@ -397,10 +397,11 @@ abstract class Enterprise_Search_Model_Adapter_Abstract
}
$attribute->setStoreId($storeId);
-
+ $preparedValue = '';
// Preparing data for solr fields
if ($attribute->getIsSearchable() || $attribute->getIsVisibleInAdvancedSearch()
|| $attribute->getIsFilterable() || $attribute->getIsFilterableInSearch()
+ || $attribute->getUsedForSortBy()
) {
$backendType = $attribute->getBackendType();
$frontendInput = $attribute->getFrontendInput();
@@ -439,16 +440,16 @@ abstract class Enterprise_Search_Model_Adapter_Abstract
if ($backendType == 'datetime') {
if (is_array($value)) {
$preparedValue = array();
- foreach ($value as &$val) {
+ foreach ($value as $id => &$val) {
$val = $this->_getSolrDate($storeId, $val);
if (!empty($val)) {
- $preparedValue[] = $val;
+ $preparedValue[$id] = $val;
}
}
unset($val); //clear link to value
$preparedValue = array_unique($preparedValue);
} else {
- $preparedValue = $this->_getSolrDate($storeId, $value);
+ $preparedValue[$productId] = $this->_getSolrDate($storeId, $value);
}
}
}
@@ -456,6 +457,7 @@ abstract class Enterprise_Search_Model_Adapter_Abstract
// Preparing data for sorting field
if ($attribute->getUsedForSortBy()) {
+ $sortValue = null;
if (is_array($preparedValue)) {
if (isset($preparedValue[$productId])) {
$sortValue = $preparedValue[$productId];
diff --git app/code/core/Enterprise/Search/Model/Adapter/Solr/Abstract.php app/code/core/Enterprise/Search/Model/Adapter/Solr/Abstract.php
index c77ed12..64e6c82 100644
--- app/code/core/Enterprise/Search/Model/Adapter/Solr/Abstract.php
+++ app/code/core/Enterprise/Search/Model/Adapter/Solr/Abstract.php
@@ -197,12 +197,13 @@ abstract class Enterprise_Search_Model_Adapter_Solr_Abstract extends Enterprise_
} else {
foreach ($facetFieldConditions as $facetCondition) {
if (is_array($facetCondition) && isset($facetCondition['from'])
- && isset($facetCondition['to'])) {
- $from = (isset($facetCondition['from']) && strlen(trim($facetCondition['from'])))
- ? $this->_prepareQueryText($facetCondition['from'])
+ && isset($facetCondition['to'])
+ ) {
+ $from = strlen(trim($facetCondition['from']))
+ ? $facetCondition['from']
: '*';
- $to = (isset($facetCondition['to']) && strlen(trim($facetCondition['to'])))
- ? $this->_prepareQueryText($facetCondition['to'])
+ $to = strlen(trim($facetCondition['to']))
+ ? $facetCondition['to']
: '*';
$fieldCondition = "$facetField:[$from TO $to]";
} else {
@@ -232,12 +233,12 @@ abstract class Enterprise_Search_Model_Adapter_Solr_Abstract extends Enterprise_
if (is_array($filters) && !empty($filters)) {
foreach ($filters as $field => $value) {
if (is_array($value)) {
- if ($field == 'price' || isset($value['from']) || isset($value['to'])) {
+ if (isset($value['from']) || isset($value['to'])) {
$from = (isset($value['from']) && !empty($value['from']))
- ? $this->_prepareFilterQueryText($value['from'])
+ ? $value['from']
: '*';
$to = (isset($value['to']) && !empty($value['to']))
- ? $this->_prepareFilterQueryText($value['to'])
+ ? $value['to']
: '*';
$fieldCondition = "$field:[$from TO $to]";
} else {
diff --git app/code/core/Enterprise/Search/Model/Catalog/Layer/Filter/Price.php app/code/core/Enterprise/Search/Model/Catalog/Layer/Filter/Price.php
index 16809fa..b3814cd 100644
--- app/code/core/Enterprise/Search/Model/Catalog/Layer/Filter/Price.php
+++ app/code/core/Enterprise/Search/Model/Catalog/Layer/Filter/Price.php
@@ -99,18 +99,18 @@ class Enterprise_Search_Model_Catalog_Layer_Filter_Price extends Mage_Catalog_Mo
}
if (!$isAuto && !empty($facets)) {
- $range = $this->getPriceRange();
+ $range = $this->getPriceRange();
}
$i = 0;
$maxIntervalsNumber = $this->getMaxIntervalsNumber();
$lastSeparator = null;
foreach ($facets as $key => $count) {
- ++$i;
- preg_match('/\[([\d\.\\\*]+) TO ([\d\.\\\*]+)\]$/', $key, $separator);
- $separator[1] = str_replace('\\*', '*', $separator[1]);
- $separator[2] = str_replace('\\*', '*', $separator[2]);
+ if (!preg_match('/\[([\d\.\*]+) TO ([\d\.\*]+)\]$/', $key, $separator)) {
+ continue;
+ }
+ ++$i;
$label = null;
$value = null;
if (isset($this->_facets[$separator[1] . '_' . $separator[2]])) {
diff --git app/code/core/Enterprise/Search/etc/config.xml app/code/core/Enterprise/Search/etc/config.xml
index 04ead70..85b7110 100644
--- app/code/core/Enterprise/Search/etc/config.xml
+++ app/code/core/Enterprise/Search/etc/config.xml
@@ -85,6 +85,15 @@
</observers>
</catalogsearch_index_process_start>
+ <catelogsearch_searchable_attributes_load_after>
+ <observers>
+ <enterprise_search>
+ <class>enterprise_search/observer</class>
+ <method>storeSearchableAttributes</method>
+ </enterprise_search>
+ </observers>
+ </catelogsearch_searchable_attributes_load_after>
+
<catalogsearch_index_process_complete>
<observers>
<enterprise_search>
@@ -126,15 +135,6 @@
</observers>
</adminhtml_catalog_product_attribute_edit_prepare_form>
- <catelogsearch_searchable_attributes_load_after>
- <observers>
- <enterprise_search>
- <class>enterprise_search/observer</class>
- <method>storeSearchableAttributes</method>
- </enterprise_search>
- </observers>
- </catelogsearch_searchable_attributes_load_after>
-
<catalogsearch_query_save_after>
<observers>
<enterprise_search>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment