Skip to content

Instantly share code, notes, and snippets.

View ruanbekker's full-sized avatar
🇿🇦

Ruan Bekker ruanbekker

🇿🇦
View GitHub Profile
@ruanbekker
ruanbekker / boto3_emr_create_cluster_with_wordcount_step.py
Created March 21, 2017 19:16
Create EMR Cluster with a Wordcount Job as a Step in Boto3
import boto3
client = boto3.client(
'emr',
region_name='eu-west-1'
)
cmd = "hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar wordcount file:///etc/services /output"
emrcluster = client.run_job_flow(
@ruanbekker
ruanbekker / python-sitemap-to-elasticsearch.py
Last active January 22, 2024 05:49
Python Script that Scrapes Sitemap and Ingest URL, Title and Tags to Elasticsearch
# centos: libxslt-devel python-devel
# debian:
import re
import time
import requests
from bs4 import BeautifulSoup
from elasticsearch import Elasticsearch
es_client = Elasticsearch(['http://10.0.1.11:9200'])
@ruanbekker
ruanbekker / 0_reuse_code.js
Created April 24, 2017 08:11
Here are some things you can do with Gists in GistBox.
// Use Gists to store code you would like to remember later on
console.log(window); // log the "window" object to the console
@ruanbekker
ruanbekker / docker-howto.txt
Created April 25, 2017 10:59
Some Docker Howto Command Line Examples
My Docker Notes. I will post more in future on details about Docker, Docker Compose, Docker Swarm on my blog under:
https://sysadmins.co.za/tag/docker
Get Docker:
# curl -sSL https://get.docker.io | bash
Docker Hub:
Search:
# docker search ubuntu
@ruanbekker
ruanbekker / nested-sitemap-to-elasticsearch.py
Created April 30, 2017 13:42
Scrapes URL, Title and Keywords from Nested Sitemap to Elasticsearch
import time
import requests
from bs4 import BeautifulSoup
from elasticsearch import Elasticsearch
es_client = Elasticsearch(['http://search-domain:9200'])
drop_index = es_client.indices.create(index='myindex', ignore=400)
create_index = es_client.indices.delete(index='myindex', ignore=[400, 404])
@ruanbekker
ruanbekker / check-ssl-expiry.sh
Created May 10, 2017 18:13
Checks SSL Expiry Date with OpenSSL
#!/bin/bash
SERVER_NAME=""
echo | openssl s_client -servername $SERVER_NAME -connect $SERVER_NAME:443 2>/dev/null | openssl x509 -noout -dates
@ruanbekker
ruanbekker / mysql-to-csv.py
Last active October 12, 2022 11:00
Dump MySQL Data to CSV with Python
# requirement: python-mysqldb
import MySQLdb as dbapi
import sys
import csv
QUERY='SELECT * FROM mydb.people;'
db=dbapi.connect(host='localhost',user='root',passwd='password')
cur=db.cursor()
@ruanbekker
ruanbekker / setup-elasticsearch-node-centos.sh
Created June 5, 2017 20:33
Bootstrap Script to Provision Elasticsearch Node on CentOS 7 fo ES Cluster
#!/bin/bash
# my testing was done with LXD/LXC so I had to set this on my hypervisor
# sysctl -w vm.max_map_count=262144
# set elasticsearch cluster name:
ES_CLUSTER_NAME="myescluster"
# set elasticsearch nodes ip/dns that will be part of the elasticsearch cluster
ES_NODES='["10.0.0.2", "10.0.0.3"]'
@ruanbekker
ruanbekker / flask-client-ip.py
Created June 7, 2017 20:42
Python Flask Show Client IP
from flask import Flask, request, jsonify
app = Flask(__name__)
@app.route('/ip', methods=['GET'])
def name():
return request.environ.get('HTTP_X_REAL_IP', request.remote_addr)
#return jsonify({'ip': request.remote_addr}), 200
#return jsonify({'ip': request.environ['REMOTE_ADDR']}), 200
@ruanbekker
ruanbekker / flask-restful-api.py
Created June 11, 2017 22:45
Basic API Server with Flask-Restful
from flask import Flask
from flask_restful import Resource, Api
app = Flask(__name__)
api = Api(app)
class Colors(Resource):
def get(self):
return {
'colors': [