Skip to content

Instantly share code, notes, and snippets.

View revolutionisme's full-sized avatar

Biplob Biswas revolutionisme

View GitHub Profile
@revolutionisme
revolutionisme / FlinkCEPKafka.java
Created June 6, 2017 13:19
For reproducing no alert generation in FlinkCEP
import ch.qos.logback.classic.Level;
import ch.qos.logback.classic.Logger;
import org.apache.flink.api.java.functions.KeySelector;
import org.apache.flink.cep.CEP;
import org.apache.flink.cep.PatternSelectFunction;
import org.apache.flink.cep.PatternStream;
import org.apache.flink.cep.PatternTimeoutFunction;
import org.apache.flink.cep.pattern.Pattern;
import org.apache.flink.cep.pattern.conditions.IterativeCondition;
import org.apache.flink.cep.pattern.conditions.SimpleCondition;
@revolutionisme
revolutionisme / Input.txt
Created June 6, 2017 13:20
Input for sending test data over kafka
1,a,1,0
5,a,3,2
6,a,2,1
8,b,5,1
12,b,4,6
@revolutionisme
revolutionisme / FlinkCEPWithKafka.java
Created June 16, 2017 13:42
Not working with event time properly, working with processing time (last event missing unless a subsequent event is thrown)
package com.airplus.poc;
import com.airplus.poc.helper.CEPRule;
import com.airplus.poc.helper.Configure;
import com.airplus.poc.helper.StringSerializerToEvent;
import com.airplus.poc.model.BAMEvent;
import org.apache.flink.api.common.ExecutionConfig;
import org.apache.flink.api.java.functions.KeySelector;
import org.apache.flink.cep.CEP;
import org.apache.flink.cep.PatternSelectFunction;
package com.airplus.poc.helper;
import com.airplus.poc.model.BAMEvent;
import org.apache.flink.cep.pattern.Pattern;
import org.apache.flink.cep.pattern.conditions.IterativeCondition;
import org.apache.flink.cep.pattern.conditions.SimpleCondition;
import org.apache.flink.streaming.api.windowing.time.Time;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@revolutionisme
revolutionisme / TypeInformation for Tuples
Last active July 28, 2017 12:45
Getting TypeInformation for Tuples
To get TypeInfomation of a Tuple with types -
TypeInformation.of(new TypeHint<Tuple3<String, List<String>, String[]>>() {})
@revolutionisme
revolutionisme / txnModel.java
Created August 22, 2017 15:40
The model which is used for serializing with avro and storing the serialized data to RocksDB
package com.airplus.poc.flink.model;
import com.airplus.poc.generated.xjc.RecordReadEventType;
import java.util.HashSet;
/**
* @author Biplob Biswas on 22.08.2017.
*/
public class TransactionStateModel {
private RecordReadEventType recordReadEvent;
@revolutionisme
revolutionisme / RDS-Aurora-CloudFormation-Example.yaml
Created November 6, 2018 14:08 — forked from sjparkinson/RDS-Aurora-CloudFormation-Example.yaml
A basic CloudFormation template for an RDS Aurora cluster.
---
AWSTemplateFormatVersion: 2010-09-09
Description: >
A basic CloudFormation template for an RDS Aurora cluster.
Parameters:
DatabaseUsername:
AllowedPattern: "[a-zA-Z0-9]+"
ConstraintDescription: must be between 1 to 16 alphanumeric characters.
@revolutionisme
revolutionisme / list.txt
Created November 12, 2018 14:41 — forked from shortjared/list.txt
List of AWS Service Principals
acm.amazonaws.com
apigateway.amazonaws.com
application-autoscaling.amazonaws.com
appstream.application-autoscaling.amazonaws.com
appsync.amazonaws.com
athena.amazonaws.com
autoscaling.amazonaws.com
batch.amazonaws.com
channels.lex.amazonaws.com
clouddirectory.amazonaws.com
@revolutionisme
revolutionisme / aws-cheat-sheet.txt
Created November 13, 2018 12:47
AWS Services and resources Cheat Sheet
Services - The service sample templates show you how you can use AWS CloudFormation with other AWS services.
- https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/sample-templates-services-us-west-2.html#w2ab1c23c48c13c39
@revolutionisme
revolutionisme / xml2csv.py
Created March 5, 2019 09:57
Read html table using pandas and write to csv
import pandas as pd
import os
from bs4 import BeautifulSoup
import re
import sys
for root, dirs, files in os.walk('/toplevelfolderpath'):
print(root, dirs, files)
for file in files:
if "xls" in file: