Skip to content

Instantly share code, notes, and snippets.

View DanielBerman's full-sized avatar

Daniel Berman DanielBerman

View GitHub Profile
@DanielBerman
DanielBerman / aws-ecs-logz-docker
Created October 6, 2016 10:48
Task definition for using the Logz.io Docker log collector on AWS ECS
{
"networkMode": "bridge",
"taskRoleArn": null,
"containerDefinitions": [
{
"volumesFrom": [],
"memory": null,
"extraHosts": null,
"dnsServers": null,
"disableNetworking": null,
@DanielBerman
DanielBerman / gist:176ee6f8395b6510ad11d4569fa8b971
Last active May 3, 2021 22:00
ELK on Ubuntu Quick Install
ELK on Ubuntu Quick Install
---------------------------
#Elasticsearch
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
sudo apt-get update
sudo apt-get install apt-transport-https
echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list
sudo apt-get update && sudo apt-get install elasticsearch
sudo su
vim /etc/elasticsearch/elasticsearch.yml
@DanielBerman
DanielBerman / gist:248e2916d8984c5cad15ae37d81535aa
Created January 23, 2017 12:44
Docker Logspout - Syslog - Logstash Filter
input {
syslog {
port => 5000
type => "docker"
}
}
filter {
grok {
match => { "message" => "%{SYSLOG5424PRI}%{NONNEGINT:ver} +(?:%{TIMESTAMP_ISO8601:ts}|-) +(?:%{HOSTNAME:service}|-) +(?:%{NOTSPACE:containerName}|-) +(?:%{NOTSPACE:proc}|-) +(?:%{WORD:msgid}|-) +(?:%{SYSLOG5424SD:sd}|-|) +%{GREEDYDATA:msg}" }
@DanielBerman
DanielBerman / .eb extension config for Elastic Beanstalk
Created February 9, 2017 08:38
.eb extension for Elastic Beanstalk to send logs to Logz.io with Filebeat
files:
"/etc/filebeat/filebeat.yml":
mode: "000755"
owner: root
group: root
content: |
filebeat:
# List of prospectors to fetch data.
prospectors:
-
@DanielBerman
DanielBerman / heartbeat.yml
Created March 29, 2017 11:53
Heartbeat configuration
############################# Heartbeat ######################################
# Configure monitors
heartbeat.monitors:
- type: http
# List or urls to query
urls: ["http://localhost:9200"]
# Configure task schedule
@DanielBerman
DanielBerman / apachelog.txt
Created November 1, 2017 13:00
Apache access log sample
200.183.100.141 - - [25/Nov/2016:16:17:10 +0000] "GET /wp-content/force-download.php?file=../wp-config.php HTTP/1.0" 200 3842 "http://hack3r.com/top_online_shops" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; YTB720; GTB7.2; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)"
input {
azureblob
{
storage_account_name => "yourStorageAccount"
storage_access_key => "yourStorageAccessKey"
container => "insights-logs-networksecuritygroupflowevent"
codec => "json"
# Refer https://docs.microsoft.com/en-us/azure/network-watcher/network-watcher-read-nsg-flow-logs
# Typical numbers could be 21/9 or 12/2 depends on the nsg log file types
file_head_bytes => 12
@DanielBerman
DanielBerman / docker-install
Created December 28, 2017 08:11
Install Docker and Docker Compose on Ubuntu
sudo apt-get update
sudo apt-get install \
apt-transport-https \
ca-certificates \
curl \
software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
input {
beats {
host => "localhost"
port => 5044
}
}
filter {
if [message] =~ /^#/ {
drop { }
---
apiVersion: v1
kind: ConfigMap
metadata:
name: filebeat-config
namespace: kube-system
labels:
k8s-app: filebeat
kubernetes.io/cluster-service: "true"
data: