purbon

Adding 2 nodes to an existing 3-node ZooKeeper ensemble without losing the Quorum

Since many deployments may start out with 3 nodes and so little is known about how to grow a cluster from 3 memebrs to 5 members without losing the existing Quorum, here is an example of how this might be achieved.

In this example, all 5 nodes will be running on the same Vagrant host for the purpose of illustration, running on distinct configurations (ports and data directories) without the actual load of clients.

YMMV. Caveat usufructuarius.

Step 1: Have a healthy 3-node ensemble

purbon

Options to handle wrong date formats in elaticsearch

Mapping date field without special handling of incorrect value, in this case all non joda conformant dates (ISO-8601) will be reported as incorrent and an exception will be thrown.

DELETE kafka

PUT kafka

purbon

#!/bin/bash

NUMCPUS=`grep -c processor /proc/cpuinfo`

if [ -z "$1" ] ; then
      NUMINSTANCES=2
else
      echo $1 | grep "^[0-9]*$" >/dev/null
      if [ "$?" -ne 0 ] ; then
            echo Argument $1 is not a number. Aborting.

purbon

import javax.net.ssl.SSLSocket;
import javax.net.ssl.SSLSocketFactory;
import java.io.*;

/** Establish a SSL connection to a host and port, writes a byte and
 * prints the response. See
 * http://confluence.atlassian.com/display/JIRA/Connecting+to+SSL+services
 */
public class SSLPoke {
    public static void main(String[] args) {

purbon

import java.math.BigInteger;
import java.security.PublicKey;
import java.security.PrivateKey;
import java.security.KeyFactory;
import java.security.Security;
import java.security.KeyPairGenerator;
import java.security.KeyPair;
import java.security.SecureRandom;
import java.security.spec.PKCS8EncodedKeySpec;
import java.security.spec.ECGenParameterSpec;

purbon

skywalker% rake test:core
--- jar coordinate com.fasterxml.jackson.core:jackson-databind already loaded with version 2.7.4 - omit version 2.7.3
--- jar coordinate com.fasterxml.jackson.core:jackson-core already loaded with version 2.7.4 - omit version 2.7.3
--- jar coordinate com.fasterxml.jackson.core:jackson-databind already loaded with version 2.7.4 - omit version 2.7.3
--- jar coordinate com.fasterxml.jackson.core:jackson-annotations already loaded with version 2.7.3 - omit version 2.7.0
--- jar coordinate com.fasterxml.jackson.core:jackson-core already loaded with version 2.7.4 - omit version 2.7.3
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.
Run options: exclude {:redis=>true, :socket=>true, :performance=>true, :couchdb=>true, :elasticsearch=>true, :elasticsearch_secure=>true, :export_cypher=>true, :integration=>true, :windows=>true}
.................................................................................................

purbon

#!/usr/bin/env ruby

require "resolv"
require 'jruby'


class Resolv
  class Hosts

    def initialize(filename = DefaultFileName)

purbon

skywalker% bin/rspec  ./logstash-core-event/spec/logstash/timestamp_spec.rb
--- jar coordinate com.fasterxml.jackson.core:jackson-annotations already loaded with version 2.7.1 - omit version 2.7.0
--- jar coordinate com.fasterxml.jackson.core:jackson-databind already loaded with version 2.7.1 - omit version 2.7.1-1
/Users/purbon/work/logstash/logstash-core/lib/logstash/api/commands/stats.rb:44: warning: key (SymbolNode:heap_used_in_bytes 47) is duplicated and overwritten on line 44
/Users/purbon/work/logstash/logstash-core/lib/logstash/api/commands/node.rb:34: warning: key (SymbolNode:vm_name 38) is duplicated and overwritten on line 36
Run options: exclude {:redis=>true, :socket=>true, :performance=>true, :couchdb=>true, :elasticsearch=>true, :elasticsearch_secure=>true, :export_cypher=>true, :integration=>true, :windows=>true}
...................FF.......

Failures:

purbon

skywalker% bin/rspec  ./logstash-core-event/spec/logstash/timestamp_spec.rb
--- jar coordinate com.fasterxml.jackson.core:jackson-annotations already loaded with version 2.7.1 - omit version 2.7.0
--- jar coordinate com.fasterxml.jackson.core:jackson-databind already loaded with version 2.7.1 - omit version 2.7.1-1
/Users/purbon/work/logstash/logstash-core/lib/logstash/api/commands/stats.rb:44: warning: key (SymbolNode:heap_used_in_bytes 47) is duplicated and overwritten on line 44
/Users/purbon/work/logstash/logstash-core/lib/logstash/api/commands/node.rb:34: warning: key (SymbolNode:vm_name 38) is duplicated and overwritten on line 36
Run options: exclude {:redis=>true, :socket=>true, :performance=>true, :couchdb=>true, :elasticsearch=>true, :elasticsearch_secure=>true, :export_cypher=>true, :integration=>true, :windows=>true}
...................FF.......

Failures:

purbon

purbon@slave-342015:~/logstash/qa$ vagrant snapshot push debian-8
==> debian-8: Snapshotting the machine as 'push_1465463427_1002'...
==> debian-8: Snapshot saved! You can restore the snapshot at any time by
==> debian-8: using `vagrant snapshot restore`. You can delete it using
==> debian-8: `vagrant snapshot delete`.
purbon@slave-342015:~/logstash/qa$ vagrant snapshot pop debian-8
==> debian-8: Forcing shutdown of VM...
==> debian-8: Restoring the snapshot 'push_1465463427_1002'...
==> debian-8: Deleting the snapshot 'push_1465463427_1002'...
==> debian-8: Snapshot deleted!