Skip to content

Instantly share code, notes, and snippets.

View geekpete's full-sized avatar
💭
Equilibrious

Peter Dyson geekpete

💭
Equilibrious
56 followers · 72 following
View GitHub Profile
@geekpete
geekpete / 20130416-todo.md
Created July 14, 2014 04:21 — forked from mrflip/20130416-todo.md

Next Steps

  • Measure time spend on index, flush, refresh, merge, query, etc. (TD - done)
  • Take hot threads snapshots under read+write, read-only, write-only (TD - done)
  • Adjust refresh time to 10s (from 1s) and see how load changes (TD)
  • Measure time of a rolling restart doing disable_flush and disable_recovery (TD)
  • Specify routing on query -- make it choose same node for each shard each time (MD)
  • GC new generation size (TD)
  • Warmers
  • measure before/after of client query time with and without warmers (MD)
@geekpete
geekpete / gist:12baad19da2da2cc24d4
Last active August 29, 2015 14:06
Download JDK8 tarball with a script by building your own cookie to accept the license with wget
wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u20-b26/jdk-8u20-linux-i586.tar.gz"
@geekpete
geekpete / gist:c702e9e443a8690501c7
Last active August 29, 2015 14:10
List expiry dates of all certificates in our Amazon AWS account with the aws cli
aws iam list-server-certificates --output text --query 'ServerCertificateMetadataList[*].[Expiration,ServerCertificateName]'|sort
@geekpete
geekpete / nested_aggs.sh
Last active August 29, 2015 14:13 — forked from dptesta/nested_aggs.sh
curl -XDELETE localhost:9200/nested_aggs
curl -XPUT localhost:9200/nested_aggs
curl -XPUT localhost:9200/nested_aggs/user/_mapping -d '
{
"_id" : {"index": "not_analyzed", "path" : "userId"},
"properties": {
"userId": {"type": "string", "index": "not_analyzed"},
"groups": {
"type": "nested",
"properties": {
@geekpete
geekpete / 20130416-todo.md
Last active August 29, 2015 14:13 — forked from mrflip/20130416-todo.md

Next Steps

  • Measure time spend on index, flush, refresh, merge, query, etc. (TD - done)
  • Take hot threads snapshots under read+write, read-only, write-only (TD - done)
  • Adjust refresh time to 10s (from 1s) and see how load changes (TD)
  • Measure time of a rolling restart doing disable_flush and disable_recovery (TD)
  • Specify routing on query -- make it choose same node for each shard each time (MD)
  • GC new generation size (TD)
  • Warmers
  • measure before/after of client query time with and without warmers (MD)
@geekpete
geekpete / gist:cd6d2fc166d329d902a7
Last active August 29, 2015 14:14 — forked from duydo/elasticsearch_best_practices.txt
If you want, I can try and help with pointers as to how to improve the indexing speed you get. Its quite easy to really increase it by using some simple guidelines, for example:
- Use create in the index API (assuming you can).
- Relax the real time aspect from 1 second to something a bit higher (index.engine.robin.refresh_interval).
- Increase the indexing buffer size (indices.memory.index_buffer_size), it defaults to the value 10% which is 10% of the heap.
- Increase the number of dirty operations that trigger automatic flush (so the translog won't get really big, even though its FS based) by setting index.translog.flush_threshold (defaults to 5000).
- Increase the memory allocated to elasticsearch node. By default its 1g.
- Start with a lower replica count (even 0), and then once the bulk loading is done, increate it to the value you want it to be using the update_settings API. This will improve things as possibly less shards will be allocated to each machine.
- Increase the number of machines you have so
@geekpete
geekpete / gist:dbbf6e89035f3b41d93c
Created October 29, 2015 06:25 — forked from garnaat/gist:1443559
Using get_all_instance_status method in boto
>>> import boto
>>> ec2 = boto.connect_ec2()
>>> stats = ec2.get_all_instance_status()
>>> stats
[InstanceStatus:i-67c81e0c]
>>> stat = stats[0]
>>> stat
InstanceStatus:i-67c81e0c
>>> stat.id
u'i-67c81e0c'
@geekpete
geekpete / gist:7350382
Created November 7, 2013 07:15
Set and forget command for wget on slow links. Tell wget to retry if no data comes through for x seconds, to resume the file and to retry infinitely until it gets the file.
wget http://<whateverurl>/<whateverfile> -O <localdestinationfile> --continue --read-timeout=5 --tries=0
@geekpete
geekpete / install-comodo-ssl-cert-for-nginx.rst
Created February 12, 2016 04:04 — forked from bradmontgomery/install-comodo-ssl-cert-for-nginx.rst
Steps to install a Comodo PositiveSSL certificate with Nginx.

Setting up a SSL Cert from Comodo

I use Namecheap.com as a registrar, and they resale SSL Certs from a number of other companies, including Comodo.

These are the steps I went through to set up an SSL cert.

Purchase the cert

@geekpete
geekpete / es-metrics.md
Created May 5, 2016 00:54
Metrics to Monitor

Elasticsearch has many metrics that can be used to determine if a cluster is healthy. Listed below are the metrics that are currently a good idea to monitor with the reason(s) why they should be monitored and any possible recourse for issues.

Version

Unless otherwise noted, all of the API requests work starting with 1.0.0. If a newer version is required for a given metric, then it is noted by the metric's name.

Metrics

Metrics are an easy way to monitor the health of a cluster and they can be easily accessed from the HTTP API. Each Metrics table is broken down by their source.