Skip to content

Instantly share code, notes, and snippets.

View davyngugi's full-sized avatar

davyngugi

View GitHub Profile
@davyngugi
davyngugi / aws_inspector_install
Created June 15, 2020 13:31
Linux install script for AWS inspector
#/bin/bash
wget https://inspector-agent.amazonaws.com/linux/latest/install
sudo bash install
@davyngugi
davyngugi / kannel-grok-processors
Created August 26, 2019 15:24
Example of multiple grok processosr for stuctured Kannel logs
LogEvent:
2019-08-26 15:14:40 Receive DLR [SMSC:example_smsc] [SVC:vodafone] [ACT:account_name] [BINF:] [FID:1234567890] [META:?smpp?dlr_err=%03%00%00&] [from:0800123456] [to:07712345678] [flags:-1:-1:-1:-1:1] [msg:131:id:1234567890 sub:001 dlvrd:001 submit date:1908261814 done date:1908261814 stat:DELIVRD err:000 text:Hello John Doe] [udh:0:]
%{TIMESTAMP_ISO8601:timestamp} %{DATA:msg} \[SMSC:%{DATA:smsc}\] \[SVC:%{DATA:service_name}\] \[ACT:%{DATA:account}\] .* \[FID:%{DATA:msg_id}\] .* \[from:%{DATA:from}\] \[to:%{DATA:to}\] .* submit date:%{NUMBER:submit_date} done date:%{NUMBER:done_date} stat:%{WORD:status} err:%{NUMBER:error} text:%{DATA:text}]
%{TIMESTAMP_ISO8601:timestamp} %{DATA:msg} \[SMSC:%{DATA:smsc}\] \[SVC:%{DATA:service_name}\] \[ACT:%{DATA:account}\] .* \[FID:%{DATA:msg_id}\] .* \[from:%{DATA:from}\] \[to:%{DATA:to}\] .* submit date:%{NUMBER:submit_date}
%{TIMESTAMP_ISO8601:timestamp} %{DATA:msg} \[SMSC:%{DATA:smsc}\] \[SVC:%{DATA:service_name}\] \[ACT:%{DATA:account}\] .* \[FID:%{DATA:msg_
@davyngugi
davyngugi / aws_codedeploy_git_hash.sh
Last active June 13, 2019 12:38
Grab the git commit hash from the downloaded bundle by aws codedeploy agent
#!/bin/bash
DIR=`tar -tzf bundle.tar | head -1 | cut -f1 -d"/"`
echo $DIR
IFS='-'
read -ra ADDR <<< "$DIR"
echo "${ADDR[-1]}"
@davyngugi
davyngugi / setup-awscli-codedeploy-agent.sh
Last active May 16, 2019 05:47 — forked from craigvantonder/setup-awscli-codedeploy-agent.sh
How to install AWS Code Deploy agent in Ubuntu 14.04 / 16.04 / 18.04
#!/bin/bash
# AWS CLI
#apt-get install zip -y;
#curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip";
#unzip awscli-bundle.zip;
#./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws;
#aws configure;
#AWS Access Key ID [None]: Obtained when creating user in AWS IAM
@davyngugi
davyngugi / filebeat.yml
Last active April 12, 2019 08:46
Filebeat JSON processors
filebeat.inputs:
# Each - is an input. Most options can be set at the input level, so
# you can use different inputs for various configurations.
# Below are the input specific configurations.
- type: log
# Change to true to enable this input configuration.
enabled: true
@davyngugi
davyngugi / aws_iam_users2.py
Created February 21, 2019 10:11
Iteratively retrieve all IAM users with the groups, roles and date created then save to csv file
import boto3, csv, sys
iam = boto3.client('iam', aws_access_key_id="XXXXXX",aws_secret_access_key="XXXXX")
user_list = []
max_items = 100
users = iam.list_users(MaxItems=max_items)
while users:
@davyngugi
davyngugi / aws_iam_users.py
Created January 30, 2019 08:55
Iteratively retrieve all IAM users with the groups, roles and date created then save to csv file
import boto3
iam = boto3.client('iam',aws_access_key_id="XXXX",aws_secret_access_key="XXXX")
user_list = []
users = (iam.get_account_authorization_details(Filter=['User'], MaxItems=1000))
while users['IsTruncated']:
marker = users['Marker']
for user_detail in users['UserDetailList']:
@davyngugi
davyngugi / gist:d0f763e667c82b91309af3795fc01ff4
Created January 2, 2019 12:49
nginx-access-custom pipeline
`PUT /_ingest/pipeline/filebeat-6.4.2-nginx-access-custom`
{
"description": "Pipeline for parsing Nginx access logs. Requires the geoip and user_agent plugins.",
"processors": [
{
"grok": {
"field": "message",
"patterns": [