Skip to content

Instantly share code, notes, and snippets.

View sgireddy's full-sized avatar

Shashi Gireddy sgireddy

View GitHub Profile
@sgireddy
sgireddy / README.md
Created May 12, 2017 06:19 — forked from kmader/README.md
Beating Serialization in Spark

Serialization

As all objects must be Serializable to be used as part of RDD operations in Spark, it can be difficult to work with libraries which do not implement these featuers.

Java Solutions

Simple Classes

For simple classes, it is easiest to make a wrapper interface that extends Serializable. This means that even though UnserializableObject cannot be serialized we can pass in the following object without any issue

public interface UnserializableWrapper extends Serializable {
 public UnserializableObject create(String parm1, String parm2);