Skip to content

Instantly share code, notes, and snippets.

View flinox's full-sized avatar
💭
"I don't want to believe. I want to know" - Carl Sagan

Fernando Lino D.T. Silva flinox

💭
"I don't want to believe. I want to know" - Carl Sagan
1 follower · 1 following
View GitHub Profile
@flinox
flinox / kafka_create_topic.sh
Created February 19, 2019 16:40
Kafka: To create a topic kafka
# partitions, how many files to store messages of topics
# replication-factor, how many copies of topics must store ( Ex.: number of nodes )
kafka-topics --zookeeper <zookeeper>:2181 --create --topic <topic-name> --partitions 3 --replication-factor 3
@flinox
flinox / consume_message_avro_through_api.sh
Last active February 19, 2019 10:57
Kafka: Consume message in avro format of a kafka topic with api of confluent REST proxy
-- CREATE CONSUMER
curl --request POST \
--url http://{{hostname_rest_proxy}}:8082/consumers/consumer-group-x \
--header 'content-type: application/vnd.kafka.v2+json' \
--data '{
"name": "client_x",
"format": "avro",
"auto.offset.reset": "latest",
"auto.commit.enable": "false"
}'
@flinox
flinox / produce_message_avro_through_api.sh
Last active December 6, 2020 06:55
Kafka: Produce message in avro format on a kafka topic through confluent REST proxy
-- PRODUCE MESSAGE AVRO
curl --request POST \
--url http://{{hostname_rest_proxy}}:8082/topics/topic-name \
--header 'accept: application/vnd.kafka.v2+json, application/vnd.kafka+json, application/json' \
--header 'content-type: application/vnd.kafka.avro.v2+json' \
--data '{
"value_schema": "{\"namespace\":\"br.com.flinox\",\"name\":\"course\",\"type\":\"record\",\"doc\":\"This schema defines a course\",\"aliases\":[\"class\",\"school\"],\"fields\":[{\"name\":\"course_source\",\"doc\":\"The system source of course\",\"type\":\"string\"},{\"name\":\"course_code\",\"doc\":\"The code of the course\",\"type\":\"int\"},{\"name\":\"course_name\",\"doc\":\"Name of the course\",\"type\":\"string\"},{\"name\":\"course_type\",\"doc\":\"Type of the course\",\"type\":\"string\"}]}",
"records": [
{
"value": {"course_source": "system_A", "course_code": 10, "course_name": "Information Systems", "course_type": "G"}
@flinox
flinox / analyze_v$log_history.sql
Created February 15, 2019 15:04
Oracle: Analyze database statistics based on v$log_history
-- Source: https://www.dba-scripts.com/scripts/diagnostic-and-tuning/troubleshooting/database-activity-log_history/
-- You can adapt the query to your needs, you just have to change the way you format the date to be able to drilldown to the precision you want.
-- Oracle PL/SQL
select round(avg(LOG_SWITCHES)) LOG_SWITCHES, DAY
from (
select to_char(trunc(first_time), 'Day') DAY, TRUNC(FIRST_TIME, 'DDD'), count(*) LOG_SWITCHES
from v$log_history
group by TRUNC(FIRST_TIME, 'DDD'), to_char(trunc(first_time), 'Day')
order by 2
@flinox
flinox / select_state_table_in_time.sql
Last active February 15, 2019 15:41
Oracle: Select the state of table from a specific moment on time
-- To know what time limit in time to get the state of table on Oracle
-- In this case, 60000 seconds, ~ 18 hrs
show parameter undo;
-- NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
-- undo_management string AUTO
-- undo_retention integer 60000
SELECT * FROM OWNER.TABLE_NAME
@flinox
flinox / generate_schema_from_avsc_file.sh
Last active February 19, 2019 10:58
Kafka: Generate a schema in confluent schema registry from an avsc file
curl -XPOST -d"{\"schema\":$(jq tostring TOPIC_NAME-value.avsc)}" http://KAFKA_BROKER:31081/subjects/topic-name-value/versions
@flinox
flinox / generate_avsc_file_from_schema_registry.sh
Last active February 21, 2019 19:10
Kafka: Generate an avsc file of schema from confluent schema registry
curl http://KAFKA_BROKER:8081/subjects/TOPIC_NAME-value/versions/latest | jq -r '.schema|fromjson' > TOPIC_NAME-value.avsc
# another way
curl http://AFKA_BROKER:8081/subjects/TOPIC_NAME-value/versions/latest/schema > TOPIC_NAME-value.avsc
@flinox
flinox / disk_space_analysis.sh
Last active February 19, 2019 11:00
Linux: Disk space analysis
# Mostra espaço utilizado por todas
# as partições do Linux o h é de Humano
df -kh
# T Para mostrar o tipo da partição Ex.: ext4
df -hT
# Quanto cada pasta ocupa ( na pasta atual )
du
@flinox
flinox / range_amazon_azure_ip_addresses.txt
Last active February 19, 2019 11:00
Cloud: Range of amazon and azure IP addresses
-- Range de IP's da Azure e Amazon, download from:
-- AZURE
https://www.microsoft.com/en-us/download/details.aspx?id=41653
-- AMAZON
https://docs.aws.amazon.com/pt_br/general/latest/gr/aws-ip-ranges.html