Skip to content

Instantly share code, notes, and snippets.

Avatar
💭
🇫🇷 🇯🇵

François-Clément Brossard renshuki

💭
🇫🇷 🇯🇵
View GitHub Profile
@renshuki
renshuki / upsert_by_query.js
Last active Sep 11, 2020
[Elasticsearch - Node.js] Upsert by query
View upsert_by_query.js
// - Try to update matching documents using _update_by_query (https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update-by-query.html)
// - If no matching documents, index a new one with desired data
//
// Install Elasticsearch Node.js client: npm install @elastic/elasticsearch
const { Client } = require('@elastic/elasticsearch')
const client = new Client({ node: 'http://localhost:9200' })
client.updateByQuery({
@renshuki
renshuki / watcher-get-number-of-shards.md
Created Apr 8, 2020
Elasticsearch / Watcher - Watcher to retrieve number of shards from _cluster/health
View watcher-get-number-of-shards.md
{
  "trigger": {
    "schedule": {
      "interval": "30m"
    }
  },
  "input" : {
    "http" : {
      "request" : {
View keybase.md

Keybase proof

I hereby claim:

  • I am renshuki on github.
  • I am fcbrossard (https://keybase.io/fcbrossard) on keybase.
  • I have a public key ASDqj90a23IOGVKTIh8wjEk0cLLD-OgGhihG__X3o-s2gAo

To claim this, I am signing this object:

@renshuki
renshuki / watcher-transform-script-extract-duplicate.md
Created Mar 4, 2020
Elasticsearch / Watcher - Transform script to extract duplicates (doc_count > 1)
View watcher-transform-script-extract-duplicate.md
"transform": {
  "script": {
    "source": "return [ 'dups': ctx.payload.aggregations.my_buckets.buckets.stream().filter(t -> { return t.doc_count > 1 }).map(t -> { return ['key': t.key ] }).collect(Collectors.toList()) ]",
    "lang": "painless"
  }
}
@renshuki
renshuki / latency-of-curl-command.md
Created Feb 28, 2020
How to retrieve latency of a cURL command
View latency-of-curl-command.md
curl -o /dev/null -s -w 'Establish Connection: %{time_connect}s\nTTFB: %{time_starttransfer}s\nTotal: %{time_total}s\n'  https://your_endpoint_url:port
@renshuki
renshuki / kibana-how-to-set-max-memory.md
Created Feb 13, 2020
Kibana - How to set instance max memory / heap (NodeJS)
View kibana-how-to-set-max-memory.md

You'll need to set NODE_OPTIONS in your environment variables.

export NODE_OPTIONS="--max-old-space-size=2048"

(which equivalent to 2GB for example)

Please note a Kibana restart is needed

@renshuki
renshuki / logstash-split-csv-columns-multiple-events.md
Last active Jan 24, 2020
Logstash pipeline configuration - split CSV columns into multiple events
View logstash-split-csv-columns-multiple-events.md

CSV Sample

col1,col2,col3
0,1,2
3,4,5
6,7,8

Code

@renshuki
renshuki / elasticsearch-bulk-index-from-json-hits.md
Last active Jan 8, 2020
Elasticsearch - Bulk index from a JSON file hits (with jq)
View elasticsearch-bulk-index-from-json-hits.md
cat file.json | jq -c '.hits.hits[] | { index: {_index:._index, _type:._type, _id:._id}}, ._source' | curl -XPOST -H "Content-Type: application/x-ndjson" localhost:9200/_bulk --data-binary @- | jq .
@renshuki
renshuki / elasticsearch-return-json-output-single-existing-field.md
Created Nov 11, 2019
Elasticsearch - Return JSON output with a single existing field
View elasticsearch-return-json-output-single-existing-field.md
GET .monitoring-es*/_search
{
  "_source": ["node_stats.process.cpu.percent"],
  "size": 200,
  "query": {
   "exists": {
     "field": "node_stats.process.cpu.percent"
   }
  }
@renshuki
renshuki / painless_script_compare_current_date_with_index_date.md
Created Oct 16, 2019
Elasticsearch - Painless script to compare the current date with a date already indexed into a document
View painless_script_compare_current_date_with_index_date.md

Elasticsearch Painless script which aims to calculate the difference in days between a date indexed into a document and the current date.

GET days_compare/_search
{
  "script_fields": {
    "diffdate": {
      "script": {
        "lang":   "painless",
        "source": """
You can’t perform that action at this time.