Skip to content

Instantly share code, notes, and snippets.

  1. Setup SSH

    for host in y113 y114 y115 y134 y135 y136; do
      ssh root@$host bash -c "'
        ssh-keygen -t rsa
      '"
    done
    
#
# kafka global settings
#
[kafka-global]
# Protocol used to communicate with brokers.
# Type: enum value { plaintext, ssl, sasl_plaintext, sasl_ssl }
#security.protocol = PLAINTEXT
# Initial list of brokers as a CSV list of broker host or host:port.

Cluster was up and running. Ansible had previously failed while waiting for the Ambari deployment to complete. Re-ran the script and ran into this issue.

TASK [ambari_gather_facts : Ask Ambari: zookeeper_tag] *************************
ok: [ec2-35-165-60-74.us-west-2.compute.amazonaws.com] => {"_content_encoding": "gzip", "changed": false, "content": "{\n  \"href\" : \"http://ec2-35-165-60-74.us-west-2.compute.amazonaws.com:8080/api/v1/clusters/metron/hosts/ec2-52-42-251-93.us-west-2.compute.amazonaws.com/host_components/ZOOKEEPER_SERVER\",\n  \"HostRoles\" : {\n    \"cluster_name\" : \"metron\",\n    \"component_name\" : \"ZOOKEEPER_SERVER\",\n    \"desired_stack_id\" : \"HDP-2.5\",\n    \"desired_state\" : \"STARTED\",\n    \"display_name\" : \"ZooKeeper Server\",\n    \"hdp_version\" : \"HDP-2.5.3.0-37\",\n    \"host_name\" : \"ec2-52-42-251-93.us-west-2.compute.amazonaws.com\",\n    \"maintenance_state\" : \"OFF\",\n    \"public_host_name\" : \"ec2-52-42-251-93.us-west-2.compute.amazonaws.com\",\n   
TASK [metron-rpms : Copy Metron RPMs] ******************************************
changed: [node1] => (item=/Users/nallen/tmp/incubator-metron/metron-deployment/packaging/docker/rpm-docker/target/RPMS/noarch/metron-common-0.3.1-201703071322.noarch.rpm)
changed: [node1] => (item=/Users/nallen/tmp/incubator-metron/metron-deployment/packaging/docker/rpm-docker/target/RPMS/noarch/metron-data-management-0.3.1-201703071322.noarch.rpm)
changed: [node1] => (item=/Users/nallen/tmp/incubator-metron/metron-deployment/packaging/docker/rpm-docker/target/RPMS/noarch/metron-elasticsearch-0.3.1-201703071322.noarch.rpm)
changed: [node1] => (item=/Users/nallen/tmp/incubator-metron/metron-deployment/packaging/docker/rpm-docker/target/RPMS/noarch/metron-enrichment-0.3.1-201703071322.noarch.rpm)
changed: [node1] => (item=/Users/nallen/tmp/incubator-metron/metron-deployment/packaging/docker/rpm-docker/target/RPMS/noarch/metron-indexing-0.3.1-201703071322.noarch.rpm)
changed: [node1] => (item=/Users/nallen/tmp/incubator-metron/m
  • If you are testing in the "Quick Dev" environment, increase the number of slots available to at least 5. This can be done within Ambari by editing Storm's supervisor.slots.ports property.
supervisor.slots.ports = [6700, 6701, 6702, 6703, 6704]
  • Alter the Profiler topology settings at config/profiler.properties to use a 1 minute period duration. This is not necessary, but is useful for the impatient.
profiler.period.duration=1
profiler.period.duration.units=MINUTES

Number of Alerts by Host

{
  "profile": "alerts-by-host",
  "onlyif": "source.type == 'snort'",
  "foreach": "ip_src_addr",
  "init": {
    "alerts": "0"