Setup Vagrant
vagrant init suse/sles11sp3
vagrant up --provider virtualbox
vagrant ssh
Install repository
zypper ar -f http://download.opensuse.org/distribution/11.3/repo/oss/ opensuse-oss
zypper ar -f http://download.opensuse.org/distribution/11.3/repo/non-oss/ opensuse-non-oss
zypper refresh
Install dependencies
zypper install flex bison libpcap-devel zlib openssl \
libopenssl-devel python python-devel python-pip \
swig cmake ncurses-devel python-curses cyrus-sasl \
cyrus-sasl-devel cyrus-sasl-gssapi
Install Librdkafka
cd
wget https://github.com/edenhill/librdkafka/archive/v0.9.4.tar.gz -O - | tar -xz
cd librdkafka-0.9.4/
./configure --prefix=/usr --enable-sasl
make
make install
Install Bro
cd
wget https://www.bro.org/downloads/release/bro-2.4.1.tar.gz -O - | tar -xz
cd bro-2.4
./configure --prefix=/usr --with-pcap=/usr/local/pfring
make
make install
Validate the Bro install.
broctl status
Configure Bro to listen to one of the TAPs. Replace <SNIFF_IFACE> with the interface that will be used.
[logger]
type=logger
host=localhost
[manager]
type=manager
host=localhost
[proxy-1]
type=proxy
host=localhost
[worker-1]
type=worker
host=localhost
interface=<SNIFF_IFACE>
lb_method=pf_ring
lb_procs=8
pin_cpus=0,1,2,3,4,5,6,7
Install the Bro plugin. This is part of the Apache Metron source code.
cd ~/metron/metron-sensors/bro-plugin-kafka/
./configure --bro-dist=/root/bro-2.4.1 --install-root=/usr/lib/bro/plugins/ --with-librdkafka=/usr
make
make install
Add the following to /usr/share/bro/site/local.bro
@load Bro/Kafka/logs-to-kafka.bro
redef Kafka::logs_to_send = set(HTTP::LOG, DNS::LOG);
redef Kafka::topic_name = "bro";
redef Kafka::tag_json = T;
redef Kafka::kafka_conf = table( ["metadata.broker.list"] = "<KAFKA-BROKER-HOST:PORT>"
, ["security.protocol"] = "SASL_PLAINTEXT"
, ["sasl.kerberos.keytab"] = "<PATH-TO-KEYTAB>"
, ["sasl.kerberos.principal"] = "<KERB-PRINCIPAL>"
, ["debug"] = "metadata"
);
Adjust logging in the following config file: /usr/etc/broctl.cfg.
# Rotation interval in seconds for log files on manager (or standalone) node.
# A value of 0 disables log rotation.
LogRotationInterval = 3600
# Expiration interval for archived log files in LogDir. Files older than this
# will be deleted by "broctl cron". The interval is an integer followed by
# one of these time units: day, hr, min. A value of 0 means that logs
# never expire.
LogExpireInterval = 7 day
# Location of the log directory where log files will be archived each rotation
# interval.
LogDir = /data1/bro/logs
# Location of the spool directory where files and data that are currently being
# written are stored.
SpoolDir = /data1/bro/spool
Make sure the changes are installed.
broctl stop
broctl deploy
Ensure that Bro logs are being generated. There should be a number of files like http.log and dns.log in this directory.
ls -ltr /data1/bro/logs
Ensure that messages are hitting Kafka.
kafka-simple-consumer-shell.sh \
--broker-list y136:6667
--topic bro \
--security-protocol SASL_PLAINTEXT \
--partition 0 \
--offset -1