Scale changes everything. Number of connections and destinations went from dozen to thousands, number of messages increased by order of magnitude. What once was quite adequate for enterprise messaging can't scale to support "Internet of Things". We need new protocols, patterns and architectures to support this new world. This session will start with basic introduction to the concept of Internet of things. Next it will discuss general technical challenges involved with the concept and explain why it is becoming mainstream now. Now we're ready to start talking about solutions. We will introduce some messaging patterns (like telemetry and command/control) and protocols (such as MQTT and AMQP) used in these scenarios. Finally we will see how Apache ActiveMQ is gearing up for this race. We will show tips for horizontal and vertical scaling of the broker, related projects that can help with deployments and what the future development road map looks like.
2. Presenters
Martyn
Taylor
• Senior
So4ware
Engineer
at
Red
Hat
• Apache
Ac>veMQ
CommiCer
• Mainly
working
on
Apache
Artemis
• Keen
interest
in
IoT
Bosanac
Dejan
• Senior
So4ware
Engineer
at
Red
Hat
• Apache
Ac>veMQ
commiCer
and
PMC
member
• Co-‐author
of
Ac>veMQ
in
Ac>on
16. JMS
• Addresses
majority
of
requirements
• 1-‐1,
1-‐Many,Grouping,Device
address
• Queues,
Topics,
Message
Selectors
• Push
• Req/Resp
• Reply
to
queues
• Inter-‐operable
• JVM
• Connec>on
failures
• Durable
Messages
and
Various
ACK
Modes
17. JMS
• Inter-‐operability
• Java
Run>me
only
• Protocol
vs
API
• Vendor
specific
• Rich
feature
set
• Complexity
on
the
client
• Large
footprint
• Vendor
protocols
• High
network
overhead
18. MQTT
• OASIS
standard
(v3.1.1)
• Created
by
IBM
• Light
weight
• Low
network
message
• Simple
protocol
• Pub
/
Sub
• Quality
of
Service
• Connec>on
failures
• Very
popular
in
IoT
scenarios
23. Subscrip>ons
• Topics
• Hierarchical
addresses
• e.g.
building/core/floor/2/office/1
• Supports
wild
cards
• building/core/floor/#
• building/core/floor/+/toilet/+
• QoS
per
topic
• Outlives
a
connec>on
24. AMQP
1.0
• Interna>onal
Standard
(ISO/IEC
ISO
19464)
• Binary
Protocol
• Rich
feature
set:
• conversa>on
mul>plexing
• advanced
flow
control
• Type
system
• QoS
Guarantees
• Symmetrical
message
exchange
• No
Broker
required
25. QoS
Guarantees
Quality
of
Service
guarantees
• At
most
once
• Fire
and
Forget
• At
least
once
• Retry
• Exactly
once
• 3
Way
Ack
26. Type
System
• Highly
Interoperable
• Rich
Type
Set
• Primi>ves
• integer,
long
• lists,
maps
• Descrip>ve
types
• Allow
applica>on
defined
types
• Mappings
in
most
languages
28. Flow
control
and
Mul>
Channels
• Limit
producer
and
consumer
flow
• Each
channel
(link)
can
be
controlled
individually
• Limit
telemetry
flow
for
command
messages
29. Protocols
Overview
• Lots
of
IoT
protocol
choices
including
several
not
discussed
here
• The
right
choice
depends
of
the
scenario
• Scenarios
may
combine
protocols
CoAP
Target
usecase Long-‐haul
(&
local)
Long-‐haul Local
(&
long-‐
haul)
Long-‐haul
Compactness High Medium Highest Verbose
Security SSL SSL DTLS SSL
Flow
control No Yes No No
Structured
message
No Yes No Yes
Complexity Medium High Low Lowest
31. Apache ActiveMQ
• Apache
Ac>veMQ
is
a
high-‐performance,
scalable
messaging
broker
• Started
as
an
open
source
JMS
broker
• Moved
to
Apache
So4ware
Founda>on
in
2006
• Now
widely
used
open
source
messaging
system
32. Apache ActiveMQ
• Mul>-‐Protocol/Mul>-‐Language
Support
• Invented
Stomp
protocol
in
2006
• Now
supports
AMQP
1.0,
MQTT
3.1.1,
HTTP,
STOMP
protocols
• Broad
range
of
supported
client
libraries
for
all
kinds
of
device
hardware
33. Apache ActiveMQ
• Benefits
• Feature
rich
• BaCle-‐tested
in
many
produc>on
environments
• Good
security
support
• Flexible
• Configurable
• Embeddable
34. Apache ActiveMQ
• Limita>ons
• Broker
core
gepng
old
• WriCen
using
synchronous
thread-‐locking
model
• It
limits
ver>cal
scalability
of
the
broker
• Number
of
threads
raise
with
number
of
connec>ons
and
des>na>ons
35. Ac>veMQ
Artemis
• Mul>
Protocol
Broker
• AMQP,
MQTT,
STOMP,
OpenWire,
Artemis
Core
• JMS
(API)
• Started
as
HornetQ
JBoss
project
in
2009
• Embedded
WildFly
(JBoss
AS)
JMS
messaging
service
• In
2014
donated
to
Apache
Ac>veMQ
• Sub
project
Ac>veMQ
Artemis.
• Latest
Release
1.1.0
36. Ac>veMQ
Artemis:
Core
• Contains
Broker
Business
logic
• Core
maintains
a
>ght
scope
• Message
Rou>ng
• Persistence
• Protocol
u>lity
API
• HA
and
Scaling
• Highly
Performance
• Protocols
are
plugged
in
38. Apache
Artemis
Summary
• Great
performance
due
to
• Reac>ve
Architecture
• Efficient
Append
only
Journal
• Mul>
-‐
Protocol
Broker
• AMQP,
MQTT,
STOMP,
CORE,
OpenWire
• JMS
• HA
and
Scalability
built
in
39. Apache
Artemis
Next
Steps
• Take
features
from
Ac>veMQ
• Enhance
OpenWire
support
for
Ac>veMQ
compa>bility
• JDBC
Journal
implementa>on
• More
protocols
• CoAP
• Your
protocol
here….
40. IoT Messaging Scaling
• Ver>cal
and
Horizontal
scaling
of
Brokers
• Qpid
Dispatch
Router
• Scalable
Deployment
42. Broker – Horizontal Scaling
• One
broker
can
only
do
so
much
• Horizontal
scaling
by
using
networks
of
brokers
• Load
balance
connec>ons
• Limita>ons
• All
des>na>ons
on
all
brokers
• Broker
network
is
the
boCleneck
43. Qpid Dispatch Router
• Lightweight
AMQP
1.0
message
router
wriCen
in
C
• hCp://qpid.apache.org/components/dispatch-‐router/
• Provides
flexible
and
scalable
interconnect
between
AMQP
endpoints
44. Qpid Dispatch Router
• It
is
not
a
broker
• It
never
owns
a
message
• It
propagates
AMQP
transfer,
seClement
and
disposi>on
frames
between
endpoints
• Message
based
or
link
based
rou>ng
Router/device1
/device2
45. Qpid Dispatch Router
• It
can
be
deployed
in
mul>ple
router-‐broker-‐endpoint
topology
• Redundant
paths
Router
Broker
Router
48. Qpid dispatch router
• Benefits
• BeCer
scaling
due
to
more
focused
tasks
• Smart
rou>ng
can
be
used
to
par>>on
the
traffic
• Ideal
candidate
for
gateway
into
the
system
49. Scalable deployments
• Combina>on
of
brokers
and
routers
provides
powerful
tool
box
• Brokers
should
focus
on
storing
messages
• Routers
should
do
the
rest
• Allows
for
horizontal
scaling
topologies
that
can
solve
IoT
challenges
56. Summary
• IoT
offers
a
new
set
of
problems
for
communica>on
• Messaging
is
a
good
fit
• We
need
new
tools
and
technologies
to
help
• Protocols
such
as
AMQP
and
MQTT
help
address
some
of
these
problems
• Ac>veMQ
and
Artemis
are
evolving
to
meet
IoT
demands
• New
tools
such
as
Apache
Dispatch
Router
allow
us
to
do
some
novel
things
to
help
scale
to
really
high
numbers
