Skip to content

Instantly share code, notes, and snippets.

View geekpete's full-sized avatar
💭
Equilibrious

Peter Dyson geekpete

💭
Equilibrious
View GitHub Profile
@scvalex
scvalex / pull_contacts.py
Created July 15, 2010 23:25
Pull data from Google Contacts
#!/usr/bin/python
"""Module for pulling contacts out of Google and storing them to disk
(or something).
See the official guide for a more in-depth look at GData Python
http://code.google.com/apis/contacts/docs/1.0/developers_guide_python.html
"""
import atom
@garnaat
garnaat / gist:1443559
Created December 7, 2011 16:52
Using get_all_instance_status method in boto
>>> import boto
>>> ec2 = boto.connect_ec2()
>>> stats = ec2.get_all_instance_status()
>>> stats
[InstanceStatus:i-67c81e0c]
>>> stat = stats[0]
>>> stat
InstanceStatus:i-67c81e0c
>>> stat.id
u'i-67c81e0c'
@duydo
duydo / elasticsearch_best_practices.txt
Last active June 20, 2024 09:59
Elasticsearch - Index best practices from Shay Banon
If you want, I can try and help with pointers as to how to improve the indexing speed you get. Its quite easy to really increase it by using some simple guidelines, for example:
- Use create in the index API (assuming you can).
- Relax the real time aspect from 1 second to something a bit higher (index.engine.robin.refresh_interval).
- Increase the indexing buffer size (indices.memory.index_buffer_size), it defaults to the value 10% which is 10% of the heap.
- Increase the number of dirty operations that trigger automatic flush (so the translog won't get really big, even though its FS based) by setting index.translog.flush_threshold (defaults to 5000).
- Increase the memory allocated to elasticsearch node. By default its 1g.
- Start with a lower replica count (even 0), and then once the bulk loading is done, increate it to the value you want it to be using the update_settings API. This will improve things as possibly less shards will be allocated to each machine.
- Increase the number of machines you have so
@athieriot
athieriot / gist:5532748
Last active June 19, 2021 13:32
Hipchat with IRSSI

If you are adventurous, here the process to have Hipchat working in your favourite IRC client:

@athoune
athoune / gist:5777474
Last active August 28, 2018 22:20
Pushing mails to Elastic Search for a Kibana analysis.
#!/usr/bin/env python
import sys
# Lamson is an application, but also the best way to read email without
# struggling with "battery include" libraries.
from lamson.encoding import from_string as parse_mail
from pyelasticsearch import ElasticSearch
from pyelasticsearch.exceptions import ElasticHttpNotFoundError
@bradmontgomery
bradmontgomery / install-comodo-ssl-cert-for-nginx.rst
Last active June 10, 2024 15:37
Steps to install a Comodo PositiveSSL certificate with Nginx.

Setting up a SSL Cert from Comodo

I use Namecheap.com as a registrar, and they resale SSL Certs from a number of other companies, including Comodo.

These are the steps I went through to set up an SSL cert.

Purchase the cert

@0xdevalias
0xdevalias / DevAlias-PentestEnvironmentSetup.sh
Last active August 16, 2020 00:36
My steps to setup a new pentest environment
# /dev/alias Pentest Environment Setup
# Version: 0.2 (20131211)
# Created By: Glenn 'devalias' Grant (http://devalias.net)
# License: The MIT License (MIT) - Copyright (c) 2013 Glenn 'devalias' Grant (see http://choosealicense.com/licenses/mit/ for full license text)
# TODO:
# * Option to check if tools (from this script and external) exist/are already installed and what versions
# * Eg nmap , metasploit, etc
# * Lair: https://github.com/fishnetsecurity/Lair
# * apt-get install python-pip
@bertspaan
bertspaan / README.md
Created January 2, 2014 15:28
Python script to convert DBF database file to CSV
@drorata
drorata / gist:146ce50807d16fd4a6aa
Last active June 3, 2024 06:00
Minimal Working example of Elasticsearch scrolling using Python client
# Initialize the scroll
page = es.search(
index = 'yourIndex',
doc_type = 'yourType',
scroll = '2m',
search_type = 'scan',
size = 1000,
body = {
# Your query's body
})
@garnaat
garnaat / lost_volumes.py
Last active December 12, 2020 01:21
Skew script to find all unattached EBS volumes
import skew
total_size = 0
total_volumes = 0
for volume in skew.scan('arn:aws:ec2:*:*:volume/*'):
if not volume.data['Attachments']:
total_volumes += 1
total_size += volume.data['Size']
print('%s: %dGB' % (volume.arn, volume.data['Size']))