1. Architecture of
MessagePack

2. • Efficient serialization library
• Rich data structures - compatible with JSON
• Dynamic typing
• Remote Procedure Call (RPC)
• Synchronous, Asynchronous and Callback style
• Concurrent calls with multiple servers
• Event-driven I/O
• Interface Definition Language (IDL) - compatible
with Thrift

3. 1. Compact 2. Fast
Binary-based format Zero-copy (C++)
Embed type information Stream deserialization

4. JSON MessagePack
null null c0
Integer 10 0a
Array [20] 91 14
String ”30” a2 ‘3’ ‘0’
Map {“40”:null} 81 a1 ‘4’ ‘0’ c0

5. JSON MessagePack
null null 4 bytes c0 1 byte
Integer 10 2 bytes 0a 1 byte
Array [20] 4 bytes 91 14 2 bytes
String ”30” 4 bytes a2 ‘3’ ‘0’ bytes
3
Map {“40”:null}bytes
11 5 bytes
81 a1 ‘4’ ‘0’ c0

6. Fixed length types Variable length types
Integer Raw bytes
Floating point
Array
Boolean
Map
Nil
type value type length body...
Type information

7. Type information Types
0x00 nil
0xc2 false
0xc3 true
0xca float
0xcb double
0xcc uint8
0xcd uint16
0xc0 0xce uint32
0xe0 0xcf uint64
0xdf int8
... ...

8. Type information Types
0x00 0x00 nil
0xc2 false
0xc3 true
Positive FixNum
0xca float
0xcb double
0x80
FixMap 0xcc uint8
0x90
0xa0 FixArray 0xcd uint16
0xc0 FixRaw 0xce uint32
0xe0 0xcf uint64
Negative FixNum 0xdf int8
... ...

11. It measured the elapsed time of serializing and deserializing
200,000 target objects. The target object consists of the three
integers and 512 bytes string.

12. MessagePack-RPC
Inter-process messaging library for
clients, servers and cluster applications.

13. MessagePack-RPC
Inter-process messaging library for
clients, servers and cluster applications.
Concept of Communicates with multiple
Future servers concurrently
Multithreaded
event-driven I/O

14. require 'msgpack/rpc'
client = MessagePack::RPC::Client.new(host, port)
result = client.call(:method, arg1, arg2)

15. require 'msgpack/rpc'
client = MessagePack::RPC::Client.new(host, port)
future1 = client.call_async(:methodA, arg1, arg2)
future2 = client.call_async(:methodB, arg1, arg2)
result1 = future1.get
result2 = future2.get

16. require 'msgpack/rpc'
client = MessagePack::RPC::Client.new(host, port)
client.callback(:method, arg, arg2) do |future|
result = future.get
end
client.join

17. require 'msgpack/rpc'
loop = MessagePack::RPC::Loop.new
client1 = MessagePack::RPC::Client.new(host1, port1, loop)
client2 = MessagePack::RPC::Client.new(host2, port2, loop)
future1 = client1.call_async(:methodA, arg1, arg2)
future2 = client2.call_async(:methodB, arg1, arg2)
result1 = future1.get
result2 = future2.get

18. require 'msgpack/rpc'
sp = MessagePack::RPC::SessionPool.new
session1 = sp.get_session(host1, port1)
session2 = sp.get_session(host2, port2)
future1 = session1.call_async(:methodA, arg1, arg2)
future2 = session2.call_async(:methodB, arg1, arg2)
result1 = future1.get
result2 = future2.get

19. Client
Session Loop Server
Client
Session Loop Server

20. Client
Session Server
Loop
Client
shared
Session event loop Server

21. Session Server
Session Pool
pools these
Loop connections
Session Server

22. Client
Server
Dispatcher Loop
Client

23. • Performance of the Loop is important.
• Java version uses Netty (JBoss’ I/O framework)
•Multithreaded
•Utilizes Java’s NIO
• C++ version uses mpio (Kumofs’ I/O architecture)
•Multithreaded
•Utilizes epoll or kqueue
• Ruby version uses Rev (libev for Ruby)
•Utilizes epoll or kqueue

26. The MessagePack Project
http://msgpack.sourceforge.net/