This guide sets up a non-clustered Nutch crawler, which stores its data via HBase. We will not learn how to setup Hadoop et al., but just the bare minimum to crawl and index websites on a single machine.
The plan is to create a pair of executables (ngrok and ngrokd) that are connected with a self-signed SSL cert. Since the client and server executables are paired, you won't be able to use any other ngrok to connect to this ngrokd, and vice versa.
Add two DNS records: one for the base domain and one for the wildcard domain. For example, if your base domain is domain.com, you'll need a record for that and for *.domain.com.
| cd ~ | |
| sudo apt-get update | |
| sudo apt-get install openjdk-7-jre-headless -y | |
| ### Check http://www.elasticsearch.org/download/ for latest version of ElasticSearch and replace wget link below | |
| # NEW WAY / EASY WAY | |
| wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.0.1.deb | |
| sudo dpkg -i elasticsearch-1.0.1.deb |
