xeago

We will first describe the infrastructure in use at VideofyMe and introduce new
components. After this we will discuss the changes to the infrastructure and
provide test results of this new infrastructure. We will also discuss how the
software is built up of easy to extend components.

xeago

{
  "query": {
    "filtered": {
      "query": { "match_all" },
      "filter": {
        "and": [
          { "type": { "value": "vets" } },
          { "geo_polygon": {
            "geoLocation": {
              "points": [

xeago

.
|-- bin
|   `-- *
|-- config
|   `-- *
|-- homebrew.mxcl.elasticsearch.plist
|-- lib
|   `-- *
|-- libexec
|   `-- *

xeago

curl -XPUT 'http://127.0.0.1:9200/test/?pretty=1'  -d '
{
   "settings" : {
      "number_of_shards" : 1
   }
}
'
curl -XPUT 'http://127.0.0.1:9200/test/foo/1?pretty=1'  -d '
{
   "bar" : 5,

xeago

# [Tue Jun 21 12:05:39 2011] Protocol: http, Server: 192.168.5.103:9200
curl -XPUT 'http://127.0.0.1:9200/test/?pretty=1'  -d '
{
   "mappings" : {
      "contact" : {
         "properties" : {
            "twitter" : {
               "type" : "object",
               "properties" : {
                  "profile" : {

xeago

Yesterday I upgraded our running elasticsearch cluster on a site which serves a few million search requests a day, with zero downtime. I've been asked to describe the process, hence this blogpost.

To make it more complicated, the cluster was running elasticsearch version 0.17.8 (released 6 Oct 2011) and I upgraded it to the latest 0.19.10. There have been 21 releases between those two versions, with a lot of functional changes, so I needed to be ready to roll back if necessary.

Our setup:

• elasticsearch

We run elasticsearch on two biggish boxes: 16 cores plus 32GB of RAM. All indices have 1 replica, so all data is stored on both boxes (about 45GB of data). The primary data for our main indices is also stored in our database. We have a few other indices whose data is stored only in elasticsearch, but are updated once daily only. Finally, we store our sessions in elasticsearch, but active sessions are cached in memcached.

xeago

import sys

def strxor(a, b):     # xor two strings of different lengths
    if len(a) > len(b):
        return "".join([chr(ord(x) ^ ord(y)) for (x, y) in zip(a[:len(b)], b)])
    else:
        return "".join([chr(ord(x) ^ ord(y)) for (x, y) in zip(a, b[:len(a)])])

def encrypt(key, msg):
    c = strxor(key, msg)

xeago

CHOOSE THE RIGHT WORD

When you give your opinion on something, you offer a conclusion or a judgment that, although it may be open to question, seems true or probable to you at the time (she was known for her strong opinions on women in the workplace). A view is an opinion that is affected by your personal feelings or biases (his views on life were essentially optimistic), while a sentiment is a more or less settled opinion that may still be colored by emotion (her sentiments on aging were shared by many other women approaching fifty). A belief differs from an opinion or a view in that it is not necessarily the creation of the person who holds it; the emphasis here is on the mental acceptance of an idea, a proposition, or a doctrine and on the assurance of its truth (religious beliefs; his belief in the power of the body to heal itself). A conviction is a firmly-held and unshakable belief whose truth is not doubted (she could not be swayed in her convictions), while a persuasion (in this sense) is a strong

xeago

require 'rubygems'
require 'Tire'

search = Tire.search('videos')
search.filter :term, "status" => "REMOVED"
# search.size 100
p search.results.map.all? {|v| v.status == "REMOVED" }
search.results.map.each { |v| puts v.status  }
puts search.to_curl

xeago

require 'terminal-table/import'

class SubsetSumMatrix
  class << self
    def create_empty_for(array)
      matrix = []
      header = [nil] + build_header_from(array)
      matrix << header
      array.each_with_index do |element,i|
        row = header.collect{|value| 'F'}