Skip to content

Instantly share code, notes, and snippets.

@nicoruti
Created January 31, 2015 23:38
Show Gist options
  • Save nicoruti/996995dacb97990d6db8 to your computer and use it in GitHub Desktop.
Save nicoruti/996995dacb97990d6db8 to your computer and use it in GitHub Desktop.
FST Serialize BouncyCastle Keypair
import java.math.BigInteger;
import java.security.KeyPair;
import java.security.KeyPairGenerator;
import java.security.SecureRandom;
import java.security.Security;
import java.security.spec.RSAKeyGenParameterSpec;
import org.bouncycastle.jce.provider.BouncyCastleProvider;
import org.nustaq.serialization.FSTConfiguration;
public class TestSerializeKeyFST {
// Fermat F4, largest known fermat prime
private static final BigInteger PUBLIC_EXP = new BigInteger("10001", 16);;
private static final int STRENGTH = 1024;
public static void main(String[] args) throws Exception {
// install BouncyCastle provider
Security.addProvider(new BouncyCastleProvider());
// generate a keypair
KeyPairGenerator gen = KeyPairGenerator.getInstance("RSA", "BC");
RSAKeyGenParameterSpec params = new RSAKeyGenParameterSpec(STRENGTH, PUBLIC_EXP);
gen.initialize(params, new SecureRandom());
KeyPair keyPair = gen.generateKeyPair();
FSTConfiguration fst = FSTConfiguration.createDefaultConfiguration();
// serialize
byte[] serialized = fst.asByteArray(keyPair);
// deserialize --> crash
KeyPair deserialized = (KeyPair) fst.asObject(serialized);
}
}
@nicoruti
Copy link
Author

This code throws the following exception:

Exception in thread "main" java.lang.RuntimeException: java.io.IOException: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
    at org.nustaq.serialization.util.FSTUtil.rethrow(FSTUtil.java:122)
    at org.nustaq.serialization.FSTConfiguration.asObject(FSTConfiguration.java:875)
    at TestSerializeKeyFST.main(TestSerializeKeyFST.java:37)
Caused by: java.io.IOException: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
    at org.nustaq.serialization.FSTObjectInput.readObject(FSTObjectInput.java:242)
    at org.nustaq.serialization.FSTConfiguration.asObject(FSTConfiguration.java:873)
    ... 1 more
Caused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
    at org.nustaq.serialization.util.FSTUtil.rethrow(FSTUtil.java:122)
    at org.nustaq.serialization.FSTObjectInput.readObjectCompatibleRecursive(FSTObjectInput.java:563)
    at org.nustaq.serialization.FSTObjectInput.readObjectCompatible(FSTObjectInput.java:525)
    at org.nustaq.serialization.FSTObjectInput.instantiateAndReadNoSer(FSTObjectInput.java:510)
    at org.nustaq.serialization.FSTObjectInput.readObjectWithHeader(FSTObjectInput.java:358)
    at org.nustaq.serialization.FSTObjectInput.readObjectFields(FSTObjectInput.java:659)
    at org.nustaq.serialization.FSTObjectInput.instantiateAndReadNoSer(FSTObjectInput.java:517)
    at org.nustaq.serialization.FSTObjectInput.readObjectWithHeader(FSTObjectInput.java:358)
    at org.nustaq.serialization.FSTObjectInput.readObjectInternal(FSTObjectInput.java:325)
    at org.nustaq.serialization.FSTObjectInput.readObject(FSTObjectInput.java:306)
    at org.nustaq.serialization.FSTObjectInput.readObject(FSTObjectInput.java:240)
    ... 2 more
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.nustaq.serialization.FSTObjectInput.readObjectCompatibleRecursive(FSTObjectInput.java:560)
    ... 11 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 1062
    at org.nustaq.serialization.coders.FSTStreamDecoder.readFByte(FSTStreamDecoder.java:249)
    at org.nustaq.serialization.coders.FSTStreamDecoder.readFShort(FSTStreamDecoder.java:311)
    at org.nustaq.serialization.FSTClazzNameRegistry.decodeClass(FSTClazzNameRegistry.java:156)
    at org.nustaq.serialization.coders.FSTStreamDecoder.readClass(FSTStreamDecoder.java:422)
    at org.nustaq.serialization.FSTObjectInput.readClass(FSTObjectInput.java:855)
    at org.nustaq.serialization.FSTObjectInput.readObjectWithHeader(FSTObjectInput.java:340)
    at org.nustaq.serialization.FSTObjectInput.readObjectInternal(FSTObjectInput.java:325)
    at org.nustaq.serialization.FSTObjectInput$2.readObjectOverride(FSTObjectInput.java:923)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:364)
    at org.nustaq.serialization.FSTObjectInput$MyObjectStream.readObjectOverride(FSTObjectInput.java:1222)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:364)
    at org.bouncycastle.jcajce.provider.asymmetric.rsa.BCRSAPublicKey.readObject(Unknown Source)
    ... 16 more

@RuedigerMoeller
Copy link

(Rant incoming)

BCRSAPublicKey.java from bouncy cancle chooses to serialize using the following whacky hack:

    private void readObject(
        ObjectInputStream in)
        throws IOException, ClassNotFoundException
    {
        in.defaultReadObject();
        try
        {
            algorithmIdentifier = AlgorithmIdentifier.getInstance(in.readObject());
        }
        catch (OptionalDataException e)
        {
            algorithmIdentifier = DEFAULT_ALGORITHM_IDENTIFIER;
        }
        catch (EOFException e)
        {
            algorithmIdentifier = DEFAULT_ALGORITHM_IDENTIFIER;
        }
    }

    private void writeObject(
        ObjectOutputStream out)
        throws IOException
    {
        out.defaultWriteObject();
        if (!algorithmIdentifier.equals(DEFAULT_ALGORITHM_IDENTIFIER))
        {
            out.writeObject(algorithmIdentifier.getEncoded());
        }
    }

If you look at it, it always reads two Objects from the stream but writes one OR two objects, then relies on Exception handling to recover. With FST this leads to a corrupted stream, so reading fails.

The concept of JDK's http://docs.oracle.com/javase/6/docs/api/java/io/OptionalDataException.html is broken as the detection of "optionaldata" can fail if the next Object in the stream by chance creates a byte sequence resembling a valid object.

You should probably file a bug report to bouncy castly library. In addition relying on exception handling for normal operation also is a big performance killer.

A valid implementation would write always an ALGORITHM_IDENTFIER (byte) so its deterministic if one or two objects can be read at reader side.

I have to figure out on how to fix that. The problem is, that detection of "non-object data" would require writing special tags in front of objects and native data (=waste a lot of performance to support stupid coding). Also its dirty as the bytes in the stream might form a valid object by accident (so spurious failure in rare cases). Additionally they catch silently an EOFException which can lead to extremely hard-to-spot errors in case you have to deal with corrupted streams/IO bugs.

I assume bcastles does this in order to keep backward compatibility as JDK serialization fails to provide reliable mechanisms to deal with versioning (however its not too hard to implement that manually).

@RuedigerMoeller
Copy link

Also they made OptionalDataException private, so I can't throw it from my code ... WTF

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment