Skip to content

Instantly share code, notes, and snippets.

View andsel's full-sized avatar

Andrea Selva andsel

  • Trento area
View GitHub Profile
@andsel
andsel / diff_matrix_plugins_vs_metadata_inventory.rb
Last active September 29, 2023 08:12
List plugins shipped which aren't in tier1 and tier2
#!/bin/ruby
require "json"
PLUGIN_DEFINITIONS = [
#tier1
#inputs
"logstash-input-azure_event_hubs", "logstash-input-beats", "logstash-input-elasticsearch", "logstash-input-file",
"logstash-input-generator", "logstash-input-heartbeat", "logstash-input-http", "logstash-input-http_poller",
"logstash-input-redis", "logstash-input-s3", "logstash-input-stdin", "logstash-input-syslog", "logstash-input-udp",
@andsel
andsel / UnsafeAllocationVsDirectMemory.java
Last active August 18, 2023 07:15
Experiment with Netty allocator to show that Netty's direct memory allocate with Unsafe is accounted as JDK's NIO direct buffers
///usr/bin/env jbang "$0" "$@" ; exit $?
//DEPS io.netty:netty-buffer:4.1.87.Final
//DEPS org.apache.logging.log4j:log4j-api:2.17.1
//DEPS org.apache.logging.log4j:log4j-core:2.17.1
// Run limiting the direct memory size with:
// jbang -Dlog4j.configurationFile=log4j2.properties -Dio.netty.maxDirectMemory=-1 -Dio.netty.allocator.numDirectArenas=1 -Dio.netty.allocator.numHeapArenas=0 -R-XX:MaxDirectMemorySize=8388608 UnsafeAllocationVsDirectMemory.java
import java.util.ArrayList;
import java.util.List;
@andsel
andsel / DLQSegmentCreator.java
Last active June 15, 2023 16:25
Generates empty DLQ segments files
///usr/bin/env jbang "$0" "$@" ; exit $?
import java.io.FileOutputStream;
import java.io.File;
/**
* Accepts two arguments:
* - the target path where to generate the files
* - the max DLQ segment id to create
* */
public class DLQSegmentCreator {
@andsel
andsel / instantiate_jdbc_input.rb
Created October 25, 2022 12:47
Direct instantiation of Logstash input plugin without a running Logstash
# encoding: utf-8
# Script to test JDBC input plugin in Logstash environment
#
# To run, copy script to Logstash folder and run:
#
# bin/ruby instantiate_jdbc_input.rb
################### Setup Logstash and Log4j ##################
require_relative "lib/bootstrap/environment"
@andsel
andsel / pom.xml
Created January 22, 2022 17:02
Murmur3 example
<dependency>
<groupId>commons-codec</groupId>
<artifactId>commons-codec</artifactId>
<version>1.15</version>
</dependency>
@andsel
andsel / Docker.alpine
Created October 5, 2021 10:24
Dockerfile used to verify Logstash with JDK16 under Alpine Linux
FROM adoptopenjdk/openjdk16:x86_64-alpine-jdk-16.0.1_9
WORKDIR /root
RUN adduser --disabled-password --gecos "" --home /home/logstash logstash && \
mkdir -p /usr/local/share/ruby-build && \
mkdir -p /opt/logstash && \
mkdir -p /opt/logstash/data && \
mkdir -p /mnt/host && \
chown logstash:logstash /opt/logstash
@andsel
andsel / pipeline.conf
Created July 19, 2021 09:37
Logstash ruby filter - remove leafs of a path selected by regexp
# input sample: {"name": "John", "outer": {"inner": {"leaf_1": "a leaf 1", "leaf_2": "a leaf 2", "leaf_3": "a leaf 3"}}}
input {
stdin {
codec => json
}
}
filter {
ruby {
@andsel
andsel / gist:f1f92dbab4757519bfc720650cd5e012
Created June 17, 2021 11:04
dump of .\gradlew assemble on windows
> Task :installDefaultGems FAILED
#<Class:0x71bf592a>: Command failed with status (1): [./gradlew assemble...]
create_shell_runner at C:/Users/andrea/workspace/logstash/vendor/bundle/jruby/2.5.0/gems/rake-12.3.3/lib/rake/file_utils.rb:67
sh at C:/Users/andrea/workspace/logstash/vendor/bundle/jruby/2.5.0/gems/rake-12.3.3/lib/rake/file_utils.rb:57
<main> at C:/Users/andrea/workspace/logstash/rakelib/compile.rake:34
execute at C:/Users/andrea/workspace/logstash/vendor/bundle/jruby/2.5.0/gems/rake-12.3.3/lib/rake/task.rb:273
each at org/jruby/RubyArray.java:1820
execute at C:/Users/andrea/workspace/logstash/vendor/bundle/jruby/2.5.0/gems/rake-12.3.3/lib/rake/task.rb:273
invoke_with_call_chain at C:/Users/andrea/workspace/logstash/vendor/bundle/jruby/2.5.0/gems/rake-12.3.3/lib/rake/task.rb:214
mon_synchronize at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/monitor.rb:237
@andsel
andsel / sender.rb
Last active June 2, 2021 10:28
Simple RabbitMQ sender
#run dockerized RabbitMQ
# docker run -v rabbitmq-data:/var/lib/rabbitmq --hostname my-rabbit --name rabbit_dev -p 15672:15672 -p 5672:5672 rabbitmq:3-management
# navigate to http://localhost:15672/#/queues
# login guest:guest
require 'march_hare'
conn = MarchHare.connect(:host => "localhost")
ch = conn.create_channel
exchange = ch.direct("sysmsg", :durable => true)
@andsel
andsel / checkpoint_scanner.rb
Last active March 22, 2021 17:09
Dump contents of Logstash checkpoint files
#!/usr/bin/env ruby
class CheckpointDecoder
attr_reader :path
def initialize(path)
@path = path
end