Skip to content

Instantly share code, notes, and snippets.

@thiagozs

thiagozs/messagepack.md

Created September 25, 2013 13:30
Show Gist options
  • Star 8 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save thiagozs/6699612 to your computer and use it in GitHub Desktop.
Save thiagozs/6699612 to your computer and use it in GitHub Desktop.
Download ZIP
MessagePack overview a tutorial
Raw
messagepack.md

Overview

MessagePack for Java is one implementation of MessagePack libraries in pure Java. See [jvm-serializers|https://github.com/eishay/jvm-serializers/wiki], which is one of well-known benchmarks for comparing Java libraries of data serialization.

API Reference

Install

Install from Maven2 repository

MessagePack for Java is released on Maven central repository. You can configure your pom.xmlas follows to use it:

<dependencies>
  ...
  <dependency>
    <groupId>org.msgpack</groupId>
    <artifactId>msgpack</artifactId>
    <version>${msgpack.version}</version>
  </dependency>
  ...
</dependencies>

Replace ${msgpack.version} with the current version of MessagePack for Java, which you can find on [http://repo1.maven.org/maven2/org/msgpack/msgpack/|http://repo1.maven.org/maven2/org/msgpack/msgpack/].

Install from git repository

You can get latest source code using git.

$ git clone git@github.com:msgpack/msgpack-java.git $ cd msgpack-java $ mvn package

Then you'll get the msgpack jar file in msgpack-java/target directory.

QuickStart

Make a messagepackable class

@Message enables you to serialize "public" fields in objects of your own classes like this.

import org.msgpack.MessagePack;
import org.msgpack.annotation.Message;

public class Main1 {
    @Message // Annotation
    public static class MyMessage {
        // public fields are serialized.
        public String name;
        public double version;
    }

    public static void main(String[] args) throws Exception {
        MyMessage src = new MyMessage();
        src.name = "msgpack";
        src.version = 0.6;

        MessagePack msgpack = new MessagePack();
        // Serialize
        byte[] bytes = msgpack.write(src);
        // Deserialize
        MyMessage dst = msgpack.read(bytes, MyMessage.class);
    }
}

If you want to serialize multiple objects sequentially, you can use Packer and Unpacker objects. This is because MessagePack.write(Object) and read(byte[]) method invocations create Packer and Unpacker objects every times. To use Packer and Unpacker objects, call createPacker(OutputStream) and createUnpacker(InputStream).

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;

import org.msgpack.MessagePack;
import org.msgpack.annotation.Message;
import org.msgpack.packer.Packer;
import org.msgpack.unpacker.Unpacker;

public class Main2 {
    @Message
    public static class MyMessage {
        public String name;
        public double version;
    }

    public static void main(String[] args) throws Exception {
        MyMessage src1 = new MyMessage();
        src1.name = "msgpack";
        src1.version = 0.6;
        MyMessage src2 = new MyMessage();
        src2.name = "muga";
        src2.version = 10.0;
        MyMessage src3 = new MyMessage();
        src3.name = "frsyukik";
        src3.version = 1.0;

        MessagePack msgpack = new MessagePack();
        //
        // Serialize
        //
        ByteArrayOutputStream out = new ByteArrayOutputStream();
        Packer packer = msgpack.createPacker(out);
        packer.write(src1);
        packer.write(src2);
        packer.write(src3);
        byte[] bytes = out.toByteArray();

        //
        // Deserialize
        //
        ByteArrayInputStream in = new ByteArrayInputStream(bytes);
        Unpacker unpacker = msgpack.createUnpacker(in);
        MyMessage dst1 = unpacker.read(MyMessage.class);
        MyMessage dst2 = unpacker.read(MyMessage.class);
        MyMessage dst3 = unpacker.read(MyMessage.class);
    }
}

Various types of values serialization/deserialization

Packer/Unpacker allows serializing/deserializing values of various types as follows. They enable serializing/deserializing values of various types like values of primitive types, values of primitive wrapper classes, String objects, byte[] objects, ByteBuffer objects and so on. As mentioned above, they also enable serializing/deserizing objects of your own classes annotated by @Message.

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.math.BigInteger;
import java.nio.ByteBuffer;

import org.msgpack.MessagePack;
import org.msgpack.packer.Packer;
import org.msgpack.unpacker.Unpacker;

public class Main3 {
    public static void main(String[] args) throws Exception {
        MessagePack msgpack = new MessagePack();

        //
        // Serialization
        //
        ByteArrayOutputStream out = new ByteArrayOutputStream();
        Packer packer = msgpack.createPacker(out);

        // Serialize values of primitive types
        packer.write(true); // boolean value
        packer.write(10); // int value
        packer.write(10.5); // double value

        // Serialize objects of primitive wrapper types
        packer.write(Boolean.TRUE);
        packer.write(new Integer(10));
        packer.write(new Double(10.5));

        // Serialize various types of arrays
        packer.write(new int[] { 1, 2, 3, 4 });
        packer.write(new Double[] { 10.5, 20.5 });
        packer.write(new String[] { "msg", "pack", "for", "java" });
        packer.write(new byte[] { 0x30, 0x31, 0x32 }); // byte array

        // Serialize various types of other reference values
        packer.write("MesagePack"); // String object
        packer.write(ByteBuffer.wrap(new byte[] { 0x30, 0x31, 0x32 })); // ByteBuffer object
        packer.write(BigInteger.ONE); // BigInteger object

        //
        // Deserialization
        //
        byte[] bytes = out.toByteArray();
        ByteArrayInputStream in = new ByteArrayInputStream(bytes);
        Unpacker unpacker = msgpack.createUnpacker(in);

        // to primitive values
        boolean b = unpacker.readBoolean(); // boolean value
        int i = unpacker.readInt(); // int value
        double d = unpacker.readDouble(); // double value

        // to primitive wrapper value
        Boolean wb = unpacker.read(Boolean.class);
        Integer wi = unpacker.read(Integer.class);
        Double wd = unpacker.read(Double.class);

        // to arrays
        int[] ia = unpacker.read(int[].class);
        Double[] da = unpacker.read(Double[].class);
        String[] sa = unpacker.read(String[].class);
        byte[] ba = unpacker.read(byte[].class);

        // to String object, ByteBuffer object, BigInteger object, List object and Map object
        String ws = unpacker.read(String.class);
        ByteBuffer buf = unpacker.read(ByteBuffer.class);
        BigInteger bi = unpacker.read(BigInteger.class);
    }
}

write methods provided by Packer allows serializing various types of data.

Unpacker provides deserialization methods for deserializing binary to primitive values. For example, if you want to deserialize binary to value of boolean (or int) type, you can use readBoolean (or readInt) method in Unpacker.

Unpacker also provides read methods for reference values. Its methods allow deserializing binary to values of references which types you specified as parameters. For example, if you want to deserialize binary to String (or byte[]) object, you have to describe a call of read(String.class) (or read(byte[].class)) method.

List, Map objects serialization/deserialization

To serialize generic container objects like List and Map objects, you can use Template. Template objects are pairs of serializer and deserializer. For example, to serialize a List object that has Integer objects as elements, you create the Template object by the following way.

Template listTmpl = Templates.tList(Templates.TInteger);

tList, TInteger are static method and field in Templates. A simple example of List and Map objects is shown in the following.

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.util.*;

import org.msgpack.MessagePack;
import org.msgpack.packer.Packer;
import org.msgpack.template.Template;
import org.msgpack.unpacker.Unpacker;
import static org.msgpack.template.Templates.tList;
import static org.msgpack.template.Templates.tMap;
import static org.msgpack.template.Templates.TString;

public class Main4 {
    public static void main(String[] args) throws Exception {
        MessagePack msgpack = new MessagePack();

        // Create templates for serializing/deserializing List and Map objects
        Template<List<String>> listTmpl = tList(TString);
        Template<Map<String, String>> mapTmpl = tMap(TString, TString);

        //
        // Serialization
        //

        ByteArrayOutputStream out = new ByteArrayOutputStream();
        Packer packer = msgpack.createPacker(out);

        // Serialize List object
        List<String> list = new ArrayList<String>();
        list.add("msgpack");
        list.add("for");
        list.add("java");
        packer.write(list); // List object

        // Serialize Map object
        Map<String, String> map = new HashMap<String, String>();
        map.put("sadayuki", "furuhashi");
        map.put("muga", "nishizawa");
        packer.write(map); // Map object

        //
        // Deserialization
        //

        byte[] bytes = out.toByteArray();
        ByteArrayInputStream in = new ByteArrayInputStream(bytes);
        Unpacker unpacker = msgpack.createUnpacker(in);

        // to List object
        List<String> dstList = unpacker.read(listTmpl);

        // to Map object
        Map<String, String> dstMap = unpacker.read(mapTmpl);
    }
}

Without annotations

If you cannot append @Message to classes representing objects that you want to serialize, register method enables you to serialize the objects of the classes.

MessagePack msgpack = new MessagePack();
msgpack.register(MyMessage2.class);

For example, if MyMessage2 class is included in external library, you cannot easily modify the class declaration and append @Message to it. register method allows to generate a pair of serializer and deserializer for MyMessage2 class automatically. You can serialize objects of MyMessage2 class after executing the method.

Optional fields

You can add new fields maintaining the compatibility. Use the @Optional in the new fields.

@Message
public static class MyMessage {
    public String name;
    public double version;

    // new field
    @Optional
    public int flag = 0;
}

If you try to deserialize the old version data, optional fields will be ignored.

Dynamic typing

As Java is a static typing language, the MessagePack has achieved dynamic typing with Value. Value has methods that checks its own type (isIntegerType(), isArrayType(), etc ...) and also converts to its own type (asStringValue(), convert(Template)).

import java.util.*;

import org.msgpack.MessagePack;
import org.msgpack.type.Value;
import org.msgpack.unpacker.Converter;

import static org.msgpack.template.Templates.*;

public class Main5 {
    public static void main(String[] args) throws Exception {
        // Create serialize objects.
        List<String> src = new ArrayList<String>();
        src.add("msgpack");
        src.add("kumofs");
        src.add("viver");

        MessagePack msgpack = new MessagePack();
        // Serialize
        byte[] raw = msgpack.write(src);

        // Deserialize directly using a template
        List<String> dst1 = msgpack.read(raw, tList(TString));

        // Or, Deserialze to Value then convert type.
        Value dynamic = msgpack.read(raw);
        List<String> dst2 = new Converter(dynamic).read(tList(TString));
    }
}

MessagePack-RPC for Java

API Reference

[JavaDoc of RPC|http://msgpack.org/rpc/javadoc/current/]

Install

Install from Maven2 repository

The official Maven2 repository for MessagePack Java is located here.

You could use the following pom.xml.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

  <modelVersion>4.0.0</modelVersion>
  <groupId>org.sample</groupId>
  <artifactId>sample</artifactId>
  <packaging>jar</packaging>
  <version>0.0.1</version>

  <repositories>
    <repository>
    <id>msgpack.org</id>
    <name>MessagePack Repository for Maven</name>
    <url>http://msgpack.org/maven2/</url>
    </repository>
  </repositories>

  <dependencies>
    <dependency>
      <groupId>org.msgpack</groupId>
      <artifactId>msgpack-rpc</artifactId>
      <version>0.6.1-devel</version>
    </dependency>
  </dependencies>
</project>

Install from git repository

You can get latest source code using git.

$ git clone git@github.com:msgpack/msgpack-rpc.git $ cd msgpack-rpc/java/ $ mvn package

Then you'll get the msgpack jar file in msgpack/java/target directory.

Dependency

MessagePack-RPC for Java depends on JBoss netty, for event-driven network I/O.

It is automatically downloaded when you install via Maven2.

QuickStart

First server program

This is a simple server program, which exposes the function named "hello". The function takes one integer argument, and returns it to the client.

import org.msgpack.rpc.Server;
import org.msgpack.rpc.loop.EventLoop;

public class ServerApp {
    public String hello(String msg, int a) {
        return msg;
    }

    public static void main(String[] args) throws Exception {
        EventLoop loop = EventLoop.defaultEventLoop();

        Server svr = new Server();
        svr.serve(new ServerApp());
        svr.listen(1985);

        loop.join();
    }
}

First client program

This is a simple client program, to communicate with the above server program. The program remotely calls "hello" function.

import org.msgpack.rpc.Client;
import org.msgpack.rpc.loop.EventLoop;

public class ClientApp {
    public static interface RPCInterface {
        String hello(String msg, int a);
    }

    public static void main(String[] args) throws Exception {
        EventLoop loop = EventLoop.defaultEventLoop();

        Client cli = new Client("localhost", 1985, loop);
        RPCInterface iface = cli.proxy(RPCInterface.class);

        iface.hello("hello", 1);
    }
}

Throw an error

You can send error to client using org.msgpack.rpc.Request. If "hello" function is called, the method that has org.msgpack.rpc.Request for the first args is called inside.

import org.msgpack.rpc.Server;
import org.msgpack.rpc.loop.EventLoop;
import org.msgpack.rpc.Request;

public class ServerApp {
    // Simple
     public String hello(String msg, int a) {
        return msg;
    }

    // Asynchronous and error response support
    public void hello(Request request, String msg, int a) {
        if(a > 0) {
            // send normal response
            request.sendResult(msg);
        } else {
            // send error
            request.sendError("a must  be a > 0");
        }
    }

    public static void main(String[] args) throws Exception {
        EventLoop loop = EventLoop.defaultEventLoop();

        Server svr = new Server(loop);
        svr.serve(new ServerApp());
        svr.listen(1985);

        loop.join();
    }
}

Handle an error

import org.msgpack.rpc.Client;
import org.msgpack.rpc.loop.EventLoop;
import org.msgpack.rpc.error.RemoteError;

public class ClientApp {
    public static interface RPCInterface {
        String hello(String msg, int a);
    }

    public static void main(String[] args) throws Exception {
        EventLoop loop = EventLoop.defaultEventLoop();

        Client cli = new Client("localhost", 1985, loop);
        RPCInterface iface = cli.proxy(RPCInterface.class);
        try {
            iface.hello("hello", -1);
        } catch(RemoteError e) {
            // "a must  be a > 0" is printed.
            System.out.println(e.getMessage());
        }
    }
}
@LjyYano
Copy link

LjyYano commented Apr 14, 2017

cool!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment