Dynamic Routing at 1 Million Messages per Second with Spring Integration

Dynamic Routing at 1 Million
Messages per Second
with Spring Integration
John Davies, CEO, Incept 5
@jtdavies
!
Josh Long, Spring Developer Advocate, Pivotal
@starbuxman

S P R I N G I N T E G R AT I O N 1 0 1

!2

what is “EAI”?

“

EAI encompasses approaches,
methodologies, standards, and
technologies allowing very
diverse but important systems
to share information, processes,
and behavior in support of the
core business.
-David Linthicum

Copyright © 2013 Incept5 Ltd. http://www.incept5.com

”

what is “EAI”?
§ file transfer
§ shared database
§ remote procedure call
§ messaging


what is “EAI”?


Spring Integration

n (n-1)
2

§ At the core, an embedded Message Bus
§ Also, an Application Integration
Framework
§ Support Enterprise Integration Patterns*
via the Spring programming model
§ Build on the Spring Platform

B

C


A

D

Spring Integration
§ Payload can be any object
§ Header values are stored in a read-only java.util.Map<K,V>
§ Similar to JMS messages, email mime envelopes, XMPP messages, etc.

public interface Message<T> {
MessageHeaders getHeaders();
T getPayload();
}


Spring Integration
§ Decouples producers from consumers
§ provide extension points for interceptors
§ messages may be queued and buffered

{

!
!
!
point to point
!

{

!
!
!
publish-subscribe
!

producer

send(Message)

message channel

receive(Message)

consumer

<channel id="async-p2p">
<dispatcher task-executor=”simpleAsyncThreadPool" />
<queue capacity=“10” />
</channel>

consumer
producer

send(Message)

message channel

consumer

receive(Message)

<publish-subscribe-channel
id="async-pubsub”
task-executor="someThreadPool" />

consumer

Spring Integration
transformers: convert payloads and modify headers
filter: discard messages based on test
router: determine next channel based on message and test

splitter: generate multiple messages from one

aggregator: assemble a single message from multiple messages

service activator: delegates processing to a backend service call


Spring Integration
<channel id = “customerChannel” />
!

<jms:outbound-channel-adapter
channel = “customerChannel” destination-name=“stocks”
connection-factory = “connectionFactory” />
!


jms:outbound-adapter

producer

send(Message)

jmsTemplate.send(javax.jms.Message)
ActiveMQ

Spring Integration
<channel id = “fileChannel” />
!

<file:inbound-channel-adapter channel = "fileChannel "
directory= "#{systemProperties['user.home']}/Desktop/in ">
<int:poller fixed-rate="1000"/>
</file:inbound-channel-adapter>

Document
Document
Document

new File(directory).list()


file:inbound-channel-adapter

!

Message<File> mf = fileChannel.receive()

consumer

Spring Integration
!

MessageChannel channel = ...
Message<Customer> customerMessage =
MessageBuilder.withPayload(customer)
.setHeader("customer-id", 10)
.build();
channel.send(customerMessage);


We need a good safety margin

• If we can process 200k/sec and we get a peek of 400k for an hour
(e.g. Christmas, Thanksgiving, breaking news etc.)

• We have a backlog of 200k/sec for 3600 seconds, if the volume drops to

150k/sec it will take 4 hours to catch up

• And we’ll need 360 GB of RAM to queue it! And that’s just 500 bytes/msg

!

• RabbitMQ was used at a large “fruit” vendor

• Queue sizes were critical in the planing of the deployment

!

• Even working at 500k/sec and a peak of 550k for 30 minutes
dropping to 400k

• 15 minute’s delay and 45 GB of RAM

!

• We need “REALTIME”

Where’s the problem?

• In order to do any work on a message we need to parse it

• For XML that’s a SAX parser, DOM or Java Binding (JAXB, JIBX etc.)

!

• If you’re going to store, search or deliver messages these are
the basic steps…

• Parse the message or part of the message you need

• Query and route the message or…

• Query to search the message

• - Possibly serialise and de-serialise the message

• - Possibly enrich or transform the message

• Write the message out (if it’s been bound to Java)

!

• All this takes time :-(

Java Binding

• Keeping the example simple a CSV generates something like
this… 
 

public class Row extends ComplexDataObject {!
  private static ﬁnal long serialVersionUID = 1L;!
  private String name;!
Name,Card Number,Expiry Date,Amount,Currency,Transaction Date,Commission,Vendor ID,Country!
  private String cardNumber;!
Stephen Hawkins,4325-6486-3757-2674,10/06,100,GBP,26-09-2006,2,14988603,UK!
private String expiryDate;!
Tim Berners-Lee,4724-7345-4725-7833,11/07,258,USD,21-12-2006,5,15688632,UK!
Bill Clinton,4924-7264-1264-8532,04/09,1250.6,USD,13-09-2006,15,66846035,US!
  private double amount;!
Angela Merkel,4457-4356-0087-0107,05/08,350,EUR,13-11-2006,7,93440252,DE!
Richard Branson,4724-7345-4725-7833,11/06,250,USD,22-02-2006,5,14988103,UK!
  private boolean isamountSet;! Tim Cook,4924-7264-1264-8532,04/09,12250,USD,16-09-2006,1.3,67434435,US!
Alan Turing,4325-6486-3757-2674,10/06,50,GBP,23-02-2006,1,14119663,UK
private String currency;!
  private Date transactionDate;!
  private double commission;!
  private boolean iscommissionSet;!
private long vendorID;!
  private boolean isvendorIDSet;!
private String country;


Message Parsing

• Taking a message with about 50-60 attributes and parsing

them so that they can be available to Spring Integration/Batch
is not exactly complex

!

• But providing the ability to manage the changes and dozens
of standards starts to add complexity

• Things change over time, we need to manage that change too

• We’d also like a consistent API so Java-Binding is still ideal

!

• Two relatively simple examples from the Telco and Financial

Services industry are RADIUS (rfc-2865) and FIX (from FPL)

• Others include ASN.1 & ISO-8583


Radius (rfc-2865)

• Remote Authentication Dial In User Service (RADIUS)

• 77 pages of binary spec... 
 
 
 
 
 
 
 
 
 
 


Modelling in C24 ﬁrst

•The RADIUS standard... 
 
 
 
 
 
 
 
 
 
 
 
 
 
 


What we can now do

• The spec may say that bit 13 of a 32 bit ﬁeld represents the
presence of a ﬁeld ABC (later in the message)

• Programatically we can test bit 5 with a mask 0x00002000

• Using the generated code we can simply call isAbcSet()

!

• It may say that bits 4-6 represent the version ID

• Programatically we can mask it and then shift it...

• mask 0x00000070

• shift >> 4

• Using the generated code we can simply call getVersionId()

!

• We now have a nice API that hides the complexity of the
binary implementation


Bits and Bytes to…

• We can now go from this…

!

jd-server:Radius TestData jdavies$
00000000 01 02 00 74 ea d5 7c 62
00000010 35 25 e5 8c 1a 17 00 00
00000020 35 37 30 36 32 37 38 38
00000030 31 35 39 30 36 32 35 38
00000040 33 34 32 31 30 32 30 39
00000050 34 37 30 30 34 31 38 38
00000060 af 08 07 32 33 34 33 35
00000070 4e 97 57 a8
00000074

hexdump -C radius.dat
1f d0 f6 fe a3 bf 36 4c
28 af 01 11 32 33 34 34
35 33 36 01 11 32 33 34
38 35 33 36 1f 11 33 35
34 35 35 36 38 5e 0e 34
36 37 33 1a 0d 00 00 28
06 06 00 00 00 02 37 06

!

• To being able to use it in Spring

• Or Mule, Fuse, Camel etc…


|...t..|b......6L|
|5%......(...2344|
|57062788536..234|
|159062588536..35|
|3421020945568^.4|
|47004188673....(|
|...23435......7.|
|N.W.|

To Spring Integration...

• Test bits and binary data in native Spring…

!

• <filter input-channel="filter-message-channel" outputchannel="process-message-channel" ref=”payload”
method=”isAbcSet"/>

!

• <filter input-channel="filter-message-channel" outputchannel="process-message-channel"
expression="payload.versionId == 5"/>


A good start but…

• We now have a pretty cool API and can use Spring but it’s
still pretty slow as it generates LOTS of objects

!

• With our telco standards binary to Java binding was “slow”

due to the number of objects, we got around 20k/sec/core

• Fine for most purposes and with an 8 core machine potentially good
for 100k/sec but we wanted better

!

• We needed a better solution, it wasn’t the parsing but the

complexity of the objects we were creating and parsing into


ByteBuffer

• Java 1.4 added java.nio.ByteBuffer

• Basically a wrapper for a byte[] 
 
 
 
 
 
 
 
 
 


Lazy parsing

• Rather than parse all of the elements from the message

every time we retrieve the elements only when we need
them

• Similar to comparing a DOM with a SAX parser

• We assume that we will only need to filter/sort/query on a limited
number of fields so we save a lot of redundant parsing

!

• Performance goes from ~20k/sec (50µs per message) to
~1 million/sec (1µS per message), some 50 times faster

• Even parsing the entire message is significantly faster


Here’s where it gets interesting...

• So we now have a Java object with a ByteBuffer holding the
message data and dozens of get() methods to get the
content

• Message call = new Message(data);

• call.getDuration();

• This works pretty fast, plus the JIT compiler kicks in after
10,000 iterations and optimises the method

bottleneck moves to the SpEL queries

• Now theinput-channel="ﬁlter-message-channel" output-channel="process-message-channel"
<ﬁlter
•

expression="payload.duration lt 0.1"/>


Reﬂection

• What’s happening under the hood is reﬂection, very powerful
but sadly still rather “slow”

• expression="payload.duration lt 0.1"

• Turns into something like...

!
!
!

• This adds about 700nS to each message

• A few of these and we’ve more than halved the performance


The Java Compiler to the rescue!

• New from Java 1.6: ToolProvider.getSystemJavaCompiler();

• We can create a generic accessor for double values...

!
!

• Can now write a class on the ﬂy that implements this
method for the getter we want e.g. getDuration() 
 
 
 


Java on the ﬂy...


Compile it...


Running it...

• Load up the compiled class (or byte-code) and run it

!
!

• Note however that the ﬁrst two lines (above) are done only
once, outside of the loop

!

• The result is “native” performance as if it was code

• Which of course it is


Back to spring...

• SpEL expressions are currently interpreted

!

• There is some basic caching

• java.lang.reflect.Field

• java.lang.reflect.Method objects are cached once discovered and
reused on subsequent evaluations

!

• But overall they are “slow” (for what we need)

• We need to look at avoiding reflection by compiling the parsed
expression into a class


SpEL compiler

• Currently proposed by Andy Clement (Spring guru) is a SpEL
compiler

• With the proposed changes SpEL has a Mixed Mode Interpreter

(MMI) system in place

• Mixed means it mixes the current interpreter with a real expression
compiler

• When an expression is evaluated X number of times SpEL compiles it to byte-code and
uses that going forward

• User does nothing, evaluations just accelerate


Does it make a difference?

• Property access:foo.bar.boo 
 
 
 
 
 
 
 
 
 
 


Does it make a difference?

• Method invocation:hello() 
 
 
 
 
 
!
!


Advantages - Performance

• We can now parse, compare and ﬁlter in around 1 µSec (1
million/sec) per core

• Make decisions on what you want early on and drop unwanted
messages

• Provide “on-the-ﬂy” aggregation

• Around 90-95% less CPU per message - $€£¥₹

!

• Disadvantages

• Less error checking

• Hardware vendors make less money

• Machine rooms become uncomfortably cold

• Power bills go down


Other Advantages

• Messages in memory are binary encoded in ByteBuffers and
so occupy a fraction of the size of a bound Java object

• Typically around 1/12 of the size, using only 8% of the memory

!

• We can queue around 10 times more messages per unit of
memory

!

• We get around 1000% capacity advantage for in-memory
grids - GigaSpaces, Hazelcast, GemFire, Coherence

!

• We can provide custom serialisation of the ByteBuffer for all
of the above - boosting node to node replication


So - In a nutshell

• We’ve gone from 20k per second to around 800k

• 40 times faster, sometimes better

!

• We’ve gone reduced memory footprint by around 12 times

• Meaning you can store 12 times more data in your grid

• Or use 1/12th of the hardware/memory

!

• You can route this with Spring at “native” JIT-compiled Java
code speeds

• So far no other framework can offer this level of efﬁciency, dynamic
ﬂexibility and performance


The Future

• SpEL to native Java compilation to be included in Spring
release

• A few issue remain to be resolved in the short term but early versions
look good

!

• Full support for standards like ASN.1

• This is a standard wire-level implementation option for ISO-20022

• ASN.1 supports multiple encoding rules, Basic, XML, Canonical, Packed
etc.

!

• XML messages encoded as packed ASN.1 (PER) would be
almost as performant as raw binary

• Complex XML as packed binary in a ByteBuffer


More Spring

• Read more on the tools we used...

!

• Spring XD - http://projects.spring.io/spring-xd/

!

• Spring Integration - http://projects.spring.io/spring-integration/

!

• Spring Batch - http://projects.spring.io/spring-batch/

!

• Spring AMQP - http://projects.spring.io/spring-amqp/


Thank you!

+
Josh Long

John Davies

Twitter: @starbuxman

Twitter: @jtdavies

GitHub: http:// github.com/joshlong

LinkedIn: http://linkedin.com/in/jdavies/

E-Mail: jlong@gopivotal.com

E-Mail: John.Davies@C24.biz

Spring & Spring Integration:  
http://spring.io

C24: http://www.c24.biz


Dynamic Routing at 1 Million Messages per Second with Spring Integration

Recomendados

Recomendados

Mais conteúdo relacionado

Mais de Joshua Long

Mais de Joshua Long (20)

Último

Último (20)

Dynamic Routing at 1 Million Messages per Second with Spring Integration