Skip to content

Instantly share code, notes, and snippets.

If you want, I can try and help with pointers as to how to improve the indexing speed you get. Its quite easy to really increase it by using some simple guidelines, for example:
- Use create in the index API (assuming you can).
- Relax the real time aspect from 1 second to something a bit higher (index.engine.robin.refresh_interval).
- Increase the indexing buffer size (indices.memory.index_buffer_size), it defaults to the value 10% which is 10% of the heap.
- Increase the number of dirty operations that trigger automatic flush (so the translog won't get really big, even though its FS based) by setting index.translog.flush_threshold (defaults to 5000).
- Increase the memory allocated to elasticsearch node. By default its 1g.
- Start with a lower replica count (even 0), and then once the bulk loading is done, increate it to the value you want it to be using the update_settings API. This will improve things as possibly less shards will be allocated to each machine.
- Increase the number of machines you have so
{
"template": "logstash-*",
"settings" : {
"number_of_shards" : 1,
"number_of_replicas" : 0,
"index" : {
"query" : { "default_field" : "@message" },
"store" : { "compress" : { "stored" : true, "tv": true } }
}
},
class percona::install {
Package {
require => Apt::Source["percona"]
}
package { "mysql":
ensure => purged,
before => Package["percona-server-server-5.5"],
}
server {
listen 81;
root /var/sites/app/webroot/;
server_name localhost;
rewrite_log on;
location ^~ /api/ {
location / {
index index.php index.html index.htm;
rewrite ^/api/(.*)$ /$1 last;
try_files $uri $uri/ /index.php?$uri&$args;
}
o = {
tfns: ['a','b', 'c'],
templates: {},
templates2: {},
loadTemplates: function () {
var t = this.tfns;
console.log('loading templates');
$.each(this.tfns, function(i,e) {
<?php
$files = scandir(__DIR__ . '/mongoose/js');
foreach ($files as $file) {
if (stripos($file, '.js') !== FALSE) {
echo '<script type = "text/javascript" src = "' . $file . '"></script>'
}
}
if (isset($_SERVER['HTTP_ORIGIN'])) {
$origin = $_SERVER['HTTP_ORIGIN'];
foreach (Configure::read('AccessControlAllowOrigin') as $domain) {
if (strpos($origin, $domain) !== FALSE) {
header("Access-Control-Allow-Origin: ".$origin);
break;
}
}
header("Access-Control-Allow-Credentials: true");
header("Access-Control-Allow-Methods: POST, GET, OPTIONS");
@rnavarro
rnavarro / python27_on_debian.sh
Created July 11, 2012 06:11 — forked from lukaslundgren/python27_on_debian.sh
How to install python 2.7 on debian
sudo apt-get install build-essential libsqlite3-dev zlib1g-dev libncurses5-dev libgdbm-dev libbz2-dev libreadline5-dev libssl-dev libdb-dev
wget http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tgz
tar -xzf Python-2.7.3.tgz
cd Python-2.7.3
./configure --prefix=/usr --enable-shared
make
sudo make install
cd ..
@rnavarro
rnavarro / cakegittmpdirs
Created July 1, 2012 06:01 — forked from tijmenb/cakegittmpdirs
Create CakePHP tmp directories with Git
# Based on: http://stackoverflow.com/questions/2218075/using-git-with-your-cakephp-project
# Procedure:
# 1. Remove app/tmp/ from .gitignore
# 2. touch app/tmp/.keep
# 3. git add app/tmp/.keep
# 4. git commit
# 5. Add app/tmp/ to .gitignore
# Clean tmpfiles and delete .gitignore