Skip to content

Instantly share code, notes, and snippets.

Avatar

dorsev dorsev

View GitHub Profile
@dorsev
dorsev / WordCountStreams.scala
Last active Nov 16, 2020
ZStream word count example
View WordCountStreams.scala
package wordCount
import java.io._
import zio._
import zio.console._
import zio.stream.{ZStream, ZTransducer}
object wordCount extends zio.App {
def run(args: List[String]) =
@dorsev
dorsev / tsconfig.json
Created Aug 15, 2020
Your first tsconfig.json
View tsconfig.json
{
"compilerOptions": {
"target": "esnext",
"module": "commonjs",
"allowJs": true,
"checkJs": false,
"outDir": "dist",
"rootDir": ".",
"strict": false,
"esModuleInterop": true /* Enables emit interoperability between CommonJS and ES Modules via creation of namespace objects for all imports. Implies 'allowSyntheticDefaultImports'. */,
@dorsev
dorsev / sentiment.analysis.metrics.list.txt
Created Mar 4, 2020
full list of metrics used in our sentiment analysis software
View sentiment.analysis.metrics.list.txt
A complete list of the metrics and names I added were:
latency metrics list
processing_duration_ms - histogram - which measures processing latencies.
deserialize_duration_ms - histogram - which measures how long it takes to deserialize a request
serialize_duration_ms - histogram - which measures how long it takes to serialize an event
store_duration_ms - histogram - which measures how long it takes to store(index/save/persist) a state change.
with all of these, I can now define a total_processing_duration_ms which will be deserialize_duration_ms + serialize_duration_ms + store_duration_ms + processing_duration_ms
Applicative usage metrics
incoming_requests_count - counter - which will indicate how many requests did we process. Using this metric we can define throughput!
@dorsev
dorsev / sentiment.analysis.service.ts
Last active Mar 6, 2020
sentiment analysis example - with metrics
View sentiment.analysis.service.ts
import SDC = require("statsd-client");
let sdc = new SDC({ host: 'localhost' });
let senService: SentimentAnalysisService; //...
while (true) {
let tweetInformation = kafkaConsumer.consume()
sdc.increment('incoming_requests_count')
let deserializedTweet: { msg: string } = deSerialize(tweetInformation)
sdc.histogram('request_size_chars', deserializedTweet.msg.length);
let sentimentResult = senService.calculateSentiment(deserializedTweet.msg)
if (sentimentResult !== undefined) {
@dorsev
dorsev / sentiment.analysis.service.ts
Last active Mar 28, 2020
Sentiment Analysis example - without metrics
View sentiment.analysis.service.ts
let senService: SentimentAnalysisService = new SentimentAnalysisService();
while (true) {
let tweetInformation = kafkaConsumer.consume()
let deserializedTweet: { msg: string } = deSerialize(tweetInformation)
let sentimentResult = senService.calculateSentiment(deserializedTweet.msg)
let seriarliedSentimentResult = serialize(sentimentResult)
sentimentStore.store(sentimentResult);
kafkaProducer.produce(seriarliedSentimentResult, 'sentiment_topic', 0);
}
@dorsev
dorsev / euler-solution-1.js
Created Jan 3, 2020
project-euler-solutions
View euler-solution-1.js
function multiplesOf3and5(number) {
return [...Array(number).keys()].filter((x) => {
if(x % 3 == 0 || x % 5 == 0){
return x;
} else return 0;
}).reduce((agg,elem) => agg+elem, 0);
}
View keyToValueSchemaInElasticsearch.sh
curl -X PUT "localhost:9200/my_index_2?pretty" -H 'Content-Type: application/json' -d'
{
"mappings": {
"_doc": {
"properties": {
"tags": {
"type": "object",
"properties": {
"keyToValue": {
"type": "keyword"
View nestedTagsSchema.sh
curl -X PUT "localhost:9200/my_index?pretty" -H 'Content-Type: application/json' -d'
{
"mappings": {
"_doc": {
"properties": {
"tags": {
"type": "nested"
}
}
}
@dorsev
dorsev / DynamicFieldQuery.sh
Last active Dec 24, 2019
query on dynamic template field
View DynamicFieldQuery.sh
curl -X GET "localhost:9200/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"match" : {
"actions.tags.name" : {
"query" : "John"
}
}
}
@dorsev
dorsev / dynamicTemplateSchema.sh
Last active Dec 24, 2019
dynamic_template schema
View dynamicTemplateSchema.sh
curl -X PUT "localhost:9200/_template/my_template?pretty" -H 'Content-Type: application/json' -d '
{
"index_patterns": [
"your-index-names*"
],
"mappings": {
"_doc": {
"dynamic_templates": [
{
"tags": {
You can’t perform that action at this time.