This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
VERSION = 0; | |
@Override | |
public void writeExternal(ObjectOutput out) throws IOException { | |
out.writeInt(VERSION); | |
out.writeInt(number); | |
out.writeObject(name); | |
} | |
@Override | |
public void readExternal(ObjectInput _in) throws IOException, ClassNotFoundException { | |
ObjectInputWrapper in = new ObjectInputWrapper(_in); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
// A wrapper classes delegating the execution of method into the container that can handle forward compatibility | |
// This class implements all the methods ObjectInput & moves the cursor | |
// whenever it sees skippable data from future versions | |
// WARNING : Currently for POC only 2 methods are implemented | |
class ObjectInputDecorator{ | |
ObjectInput input; | |
ObjectInputDecorator(ObjectInput input){ | |
this.input=input; | |
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
// Follows the template pattern to provide container for ObjectInput | |
static abstract class ObjectInputReaderTemplate<T> { | |
private ObjectInput objectInput; | |
public ObjectInputReaderTemplate(final ObjectInput objectInput){ | |
this.objectInput=objectInput; | |
} | |
public final T execute() throws IOException { | |
T t = get(); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
// IntelliJ generated | |
@Override | |
public boolean equals(Object o) { | |
if (this == o) return true; | |
if (!(o instanceof ElementNameKey)) return false; | |
ElementNameKey that = (ElementNameKey) o; | |
if (field1 != that.field1) return false; | |
if (field2 != that.field2) return false; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
__author__ = 'pmanvi' | |
import os | |
import sys | |
from email.parser import Parser | |
from elasticsearch import Elasticsearch | |
from datetime import date | |
import traceback | |
p = Parser() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
__author__ = 'pmanvi' | |
from elasticsearch import Elasticsearch | |
from elasticsearch_dsl import Search, Q | |
es = Elasticsearch([{'host': 'localhost'},{'port': 9200}]) | |
count = Search(using=es).index("enron-email").count() | |
print(count) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"aggs": { | |
"enron_size_ranges": { | |
"range": { | |
"field": "content_size_in_bytes", | |
"ranges": [ | |
{ | |
"key": "0KB to 1KB", | |
"from": 0, | |
"to": 1024 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
buckets: [ | |
{ | |
key: 0KB to 1KB | |
from: 0 | |
from_as_string: 0.0 | |
to: 1024 | |
to_as_string: 1024.0 | |
doc_count: 124181 | |
} | |
{ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
List<Integer> numbers = asList(1, 2, 3, 4, 5); | |
List<Integer> even_numbers = numbers.stream().filter(num -> num % 2 == 0).collect(Collectors.toList()); | |
System.out.println(even_numbers) | |
// will print [2,4] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
public static LocalDate getLocalDateWithUnknownFormat(String dString) { | |
LocalDate[] localDate = {null}; | |
Arrays.asList("yyyy-MM-dd'T'HH:mm:ss'Z'", "yyyy-MM-dd'T'HH:mm:ssZ", | |
"yyyy-MM-dd'T'HH:mm:ss", "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'", | |
"yyyy-MM-dd'T'HH:mm:ss.SSSZ", "yyyy-MM-dd HH:mm:ss", | |
"MM/dd/yyyy HH:mm:ss", "MM/dd/yyyy'T'HH:mm:ss.SSS'Z'", | |
"MM/dd/yyyy'T'HH:mm:ss.SSSZ", "MM/dd/yyyy'T'HH:mm:ss.SSS", | |
"MM/dd/yyyy'T'HH:mm:ssZ", "MM/dd/yyyy'T'HH:mm:ss", | |
"yyyy:MM:dd HH:mm:ss", "yyyyMMdd") | |
.stream().parallel().forEach(f-> { |
OlderNewer