Skip to content

Instantly share code, notes, and snippets.

View renshuki's full-sized avatar
💭
🇫🇷 🇯🇵 🇺🇸

Francois-Clement Brossard renshuki

💭
🇫🇷 🇯🇵 🇺🇸
View GitHub Profile
@renshuki
renshuki / kibana_err_cert_common_name_invalid.md
Created May 19, 2022 07:17
Kibana NET::ERR_CERT_COMMON_NAME_INVALID error

If you get a NET::ERR_CERT_COMMON_NAME_INVALID when trying to connect to Kibana from your browser, it means the CN and/or SAN you defined at the certificate generation is incorrect. Please refer to the Encrypt traffic between your browser and Kibana documentation for more details on how to properly generate a server certificate for Kibana.

@renshuki
renshuki / kibana-legacy-data-table-visualization.md
Created August 13, 2021 04:35
Enable Kibana legacy data table visualization (prior to EUI datagrid)

Add vis_type_table.legacyVisEnabled: true to kibana.yml

@renshuki
renshuki / logstash-output-elasticsearch-bulk-request-size-monitoring.md
Created June 25, 2021 06:03
Logstash - output.elasticsearch bulk requests size monitoring hack

Command:

tcpdump -A 'tcp port 9200 and (tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x504f5354)' -i lo0 | egrep -A 5 -i "POST /_bulk"

Expected output (w/ Content-Length):

...U...5POST /_bulk HTTP/1.1
Connection: Keep-Alive
@renshuki
renshuki / elasticsearch-adjust-scoring.md
Created June 25, 2021 03:25
Elasticsearch - Curation of resources / methods to adjust scoring in Elasticsearch
@renshuki
renshuki / logstash-log4j2-logs-delete-after.md
Created June 24, 2021 23:24
Logstash - log4j2 logs deletion after x

e.g Delete Logstash logs after 45 days

appender.rolling.strategy.type = DefaultRolloverStrategy
appender.rolling.strategy.action.type = Delete
appender.rolling.strategy.action.basepath = ${sys:ls.logs}
appender.rolling.strategy.action.condition.type = IfFileName
appender.rolling.strategy.action.condition.glob = logstash-${sys:ls.log.format}-*
appender.rolling.strategy.action.condition.nested_condition.type = IfLastModified
appender.rolling.strategy.action.condition.nested_condition.age = 45D
@renshuki
renshuki / elasticsearch-threadpool-size-details.md
Last active May 10, 2022 13:30
Elasticsearch threadpool size details using _cat/thread_pool API

Run GET _cat/thread_pool?v&h=node_name,name,max,pool_size,size,type,queue_size

Gives you details on Elasticsearch threadpool size, type, queue_size.

Example output

node_name           name                                    max pool_size size type                  queue_size
instance-0000000009 analyze                                             0    1 fixed                         16
instance-0000000009 ccr                                                32   32 fixed                        100
@renshuki
renshuki / upsert_by_query.js
Last active September 11, 2020 08:05
[Elasticsearch - Node.js] Upsert by query
// - Try to update matching documents using _update_by_query (https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update-by-query.html)
// - If no matching documents, index a new one with desired data
//
// Install Elasticsearch Node.js client: npm install @elastic/elasticsearch
const { Client } = require('@elastic/elasticsearch')
const client = new Client({ node: 'http://localhost:9200' })
client.updateByQuery({
@renshuki
renshuki / watcher-get-number-of-shards.md
Created April 8, 2020 06:12
Elasticsearch / Watcher - Watcher to retrieve number of shards from _cluster/health
{
  "trigger": {
    "schedule": {
      "interval": "30m"
    }
  },
  "input" : {
    "http" : {
 "request" : {

Keybase proof

I hereby claim:

  • I am renshuki on github.
  • I am fcbrossard (https://keybase.io/fcbrossard) on keybase.
  • I have a public key ASDqj90a23IOGVKTIh8wjEk0cLLD-OgGhihG__X3o-s2gAo

To claim this, I am signing this object:

@renshuki
renshuki / watcher-transform-script-extract-duplicate.md
Created March 4, 2020 06:45
Elasticsearch / Watcher - Transform script to extract duplicates (doc_count > 1)
"transform": {
  "script": {
    "source": "return [ 'dups': ctx.payload.aggregations.my_buckets.buckets.stream().filter(t -> { return t.doc_count > 1 }).map(t -> { return ['key': t.key ] }).collect(Collectors.toList()) ]",
    "lang": "painless"
  }
}