Session presented at the 6th IndicThreads.com Conference on Java held in Pune, India on 2-3 Dec. 2011. http://Java.IndicThreads.com

1. REST web services and Google
Protocol Buffers
Prasad Nirantar
BMC Software
1

2. Agenda
• REST concepts
• Introduction Google Protocol Buffers (GPB)
• Details of Google Protocol Buffers messages
• Demo
• Performance Comparison with other serialization
techniques
• Conclusions
2

3. REST (Representational State Transfer)
• Identifies principles why web is so
prevalent and ubiquitous
• Representational State Transfer
• Architectural style for building loosely
coupled systems
• Architectural style – General principles
informing/guiding creation of an
architecture
3

4. REST Style Principles
• Resource identification
• Uniform interface
• Self Describing messages
• Hypermedia driving application state
• Stateless transactions
4

5. Resource and its Representation
Browser
GET
http://example.org/Resource1
Resource HTTP/1.1
Accept: Application/xml
http://example.org/Resource HTTP/1.1 HTTP 200 OK
…
<Resource>
…
</Resource>
5

6. Multiple Resource representation
with single URI
XML representation
Resource
http://example.org/Resource
Text representation
6

7. Content negotiation
• Content selection
• Different types of content supported
• JSON
• XML
• A resource can give response in a
representation as per the request
7

8. Need for efficient mechanism
•Data intensive systems
•Efficient mechanism to send/receive data
8

9. Problems with XML
• Verbose
• Tends to be inefficient
9

10. Solution - Use of Binary content
Problems it tries to solve
• Performance (with Interoperability and
Portability)
Examples
• Google Protocol Buffers
• Avro
10

11. Introducing Google Protocol Buffers
• Language neutral, platform neutral,
extensible way of serializing structured
data
• Flexible, efficient, automated
• Supported languages - Java, C++, Python
• Developed and used by Google
• GPB project has BSD license
11

12. Protocol buffer messages
Message –
• Specify structure and details of the
information being serialized.
• Each protocol buffer message is a small
logical record of information, containing a
series of name-value pairs.
12

13. Details: .proto Message
• Data is hierarchically structured in messages
• Each message has one or more uniquely
numbered fields
• Each field has name and value
• Value types can be
• numbers (integer or floating-point)
• boolean
• strings,
• raw bytes
• other protocol buffer message types,
13

14. Details: .proto Message
Fields types
• Required
• Optional
• Repeated
Tag
• Unique number with each field in message
• From 1-15 take 1 byte to encode (for frequent)
• 16-2047 take 2 bytes and so on
• Smallest 1 largest is 229 -1
14

15. .proto file example
message Person {
required string name = 1;
required int32 id = 2;
optional string email = 3;
enum PhoneType { MOBILE = 0; HOME = 1; WORK = 2; }
message PhoneNumber {
required string number = 1;
optional PhoneType type = 2 [default = HOME];
} repeated PhoneNumber phone = 4;
}
15

16. Compilation of .proto messages
• A “protoc” compiler should be
downloaded first.
• From .proto file, the PB compiler
generates code of your choice of language
from -Java, C++, Python
• For Java – The compiler generates Java
classes containing POJO and Builder
• Parsing and serialization API is available
16

17. Protocol Buffer and REST
• Suits for the cases where you want to
optimize on performance and size of data
transferred
• Use the protocol buffer as a content
• Adding new MIME types for JAX-RS is easy
• The examples in demo are based on JAX-
RS
17

18. Resource & Representation-protobuf
GET
http://example.org/Resource1
Resource HTTP/1.1
Accept: Application/x-protobuf
http://example.org/Resource HTTP/1.1 HTTP 200 OK
Binary representation …
18

19. Stack - REST with GPB
Java client
Protocol buffer libraries
Protocol buffer libraries JAX-RS client libraries
JAX-RS libraries
Server component C/Python client
Protocol buffer libraries
Server
Http Client
19

20. Demo
20

21. Alternatives for binary content
• Use of Java serialized objects
• Use of AVRO
• Similar data format by Apache
• Uses JSON object in string format to describe
schema
• Compatible with C, Java, Python
21

22. Comparison - GPB vs XML
• Where GPB wins
• 3 to 10 times smaller
• 20 to 100 times faster
• Less ambiguous
• Generate data classes for easy use
• Where XML wins
• Humanly readable and editable
• A good markup choice
22

23. Comparison : serialization techniques
technology Object create Serialize Deserialize Total Time Serialized Size
java 169 25773 71574 97347 919
protobuf 471 7227 3479 11598 231
avro-generic 4024 8019 4779 12798 211
json (jackson) 176 7224 6084 13308 378
thrift 228 7303 7949 15252 353
java(Externalizable) 170 2874 3043 5917 264
JsonMarshaller 171 24618 41115 65733 370
23

24. Disadvantages of protocol buffers
• Careful with type of fields (required or
optional) for backwards compatibility
• Debugging concerns
• Not humanly readable
24

25. Conclusions
• With the JAX-RS the newer data
serialization formats can be plugged into
REST
• Useful for the data intensive operations,
in such cases performance benefits can be
achieved
• Not to be used when humanly readable
formats/markups are desired
25

26. References
Developer Guide - Protocol Buffers - Google Code
Rest in Practice – Webber, Parastaridis, Robinson Oreilly Publication
Restful Web Services : Principles, patterns, emerging technologies – Erik Wilde
Thrift-protobuf-compare Comparing various aspects of Serialization libraries on the JVM platform
(http://code.google.com/p/thrift-protobuf-compare/ )
Using JAX-RS with Protocol Buffers for high-performance REST APIs (http://www.javarants.com/)
26