1. Succeeding with the Apache SOA stack
And debunking some SOA myths
Johan Edstrom
SOA Executive and Apache developer.
jedstrom@ savoirtech.com
joed@ apache.org
Jeff Genender
Principal
jgenender@ savoirtech.com
3. Agenda
What is an ESB?
• Where would an “ESB” fit in?
• How do you choose the ESB?
• Reasons for picking the Apache ServiceMix stack
Modular
Stable
Cluster capable
How to design software and the build system for an ESB
• Karaf and ServiceMix are OSGi based, ServiceMix also can handle JBI.
• The Apache projects are modularized using Maven.
Get over it and accept it :)
4. Key parts to scalability
The right resources in the right place
Messaging
• Or any other asynchronous system
State driven
Conversational
Effectively aggregating
Transactionally safe
Correctly utilizing EE
5. W is an ESB?
hat
In my world it is simply a runtime for deploying asynchronous,
event driven code.
• So, yes; of course most of you have deployed and built ESB solutions.
• Yes, of course we are partial to “ESB” solutions.
What is an ESB in marketing slides?
• Components
• Data-transformations
• Scalability and performance (You can randomly wave your hands here)
• A Bus
This is a mythical, magical, illusionary fantastically awesome thing that
transparently just makes everything work.
6. Where does the marketing ESB fit in?
Picking up a file from a file area, transforming it with XSLT,
publishing to a web-service.
Consuming a web-service, using a RoutingSlip and sending to a
product, then inventory queue on a JMS provider
Consuming a web-service payload and transforming the data to
an insert in a JDBC component
Providing “Enterprise routing infrastructure”
Providing “Enterprise orchestration”
7. And what about a real world one?
Where you need to do any of the marketing ESB stuff
• Of course they are valid use-cases
Where you want web-apps to be able to share resources
• ServiceMix / Karaf happily will work as a servlet container
Long running processing, “Orchestration”
Any type of data transformation scenarios
8. Real one
Multi-Protocol input/output
Complex publish subscribe systems
Where you need to process in parallel
In Springism’s - anywhere you have a Bean-Ref you could start
breaking things apart.
12. So how is that “ESB” designed?
This is not far from an EE container
• Add OpenEJB if you want and you’ll have a full stack
It contains a web-services layer
Dependency injection services
OSGi integration and legacy (JBI) support
Camel for routing, transformation and mediation
Aries and Spring for EE support, JNDI, TX, JPA
Web-app deployment support
13. So why is that so cool?
Well.... You can treat it in the classical manner
Or as a really kick-ass funky container
Or - as a nice fusion of them both!
14. Apache ActiveMQ
Communications platform of choice, providing high
performance, scalability, and mission-critical reliability for
distributed enterprise computing.
Apache ActiveMQ supports based on specs:
• JMS 1.1
• J2EE 1.4 integration-related components including JDBC, JCA, and EJBs;
dependent specifications such as JTA and JNDI;
• AJAX, REST, HTTP, TCP, SSL, NIO+SSL, UDP, multicast, JGroups and JXTA
transport protocols
15. ActiveMQ “Instability”
Instability... what is it?
• Most common term used by end users
• Generally refers to ActiveMQ being frozen or completely seized
Refers to producers being unable to send message
Refers to queues being frozen/seized or stuck messages
Generally temporarily is fixed by the holdover from the Microsoft days
• The 3-finger salute (ctrl-alt-delete)
• Restart of ActiveMQ
Followed by complaints that ActiveMQ is trash
• Blog by David Pollak: “ActiveMQ: not ready for prime time”
http://goodstuff.im/activemq-not-ready-for-prime-time
• Is ActiveMQ trash or not properly written?
Not trash...its a very properly written... its extremely powerful
• If I have these problems, what do you do? Diagnose it first ;-)
16. Apache ActiveMQ
This is another presentation :)
• So you need to attend that one if you want to debate ActiveMQ.
17. Apache ActiveMQ - Additional functionality
Advisory Queues and Topics are administrative channels.
• The Broker periodically posts information about the status of known destinations
• Administration focused programs can subscribe to advisory messages like any
other Topic
Useful life-cycle and supplementary notifications
Advisories are generated for the following:
• Starting and stopping consumers and producers
• Creating and destroying temporary destinations
• Expiring messages on topics and queues
• Sending messages from brokers to destinations with no consumers
• Starting and stopping connections
18. Apache ActiveMQ, Easy to scale
Can bridge, multicast, “cluster” as well as integrate with other
JMS providers
19. Apache ActiveMQ - High Availability
Pro Con
Replicated No “single” point of Supports only a single
failure slave. Requires
manual restart and
synchronization to
bring back a failed
master
Shared File No “single” point of Requires a networked
failure. Can run file-system solution
multiple slaves, with locking
automatic recovery.
JDBC No “single” point of Slow
failure. Can run
multiple slaves,
20. Apache Camel
A powerful rule based routing and mediation engine.
Provides a POJO-based implementation of Enterprise
Integration Patterns using Java DSL (Domain Specific
Language) or Spring / Aries Blueprint to configure routing and
mediation rules.
Uses generics, annotations and URIs so that it can easily work
directly with any kind of transport or messaging model such as
HTTP, JMS, JBI, SCA, MINA or CXF Bus API without
mandating a normalized message API.
Designed as a small library which has “minimal” dependencies
for easy embedding in any Java application.
21. Apache Camel Features
Camel is a Domain Specific Language (DSL) focussed on
implementing Enterprise Integration Patterns (EIPs)
• Examples: multicast, splitter, content-based router, routing slip, dynamic routers,
aggregator
EIPs are implemented by Camel “Processors”
• Users can develop their own processors
• A processor is a fundamental building block
• Bean language and bindings exists so that not a single piece of Apache Camel Imports
will be necessary when integrating your existing code
Camel can connect to a wide variety of integration technologies
• Examples: JMS, HTTP, FTP, SOAP, File - There are ~ 120 components
• Integrations are implemented by Camel “Components”
• Users can develop their own components
22. And it is a pretty healthy library!
Camel makes application design & development easy:
• Reduces time to create integration flows
• Reduces complexity of integration flows and makes them easier to maintain
• Routes can be deployed in mainstream typical containers (OSGi, JEE, Servlet, etc.)
Used in several commercial offerings
Used in “competing” offerings
About as close to an Integrations standard as you can get
The community stays at a high level:
• 503 subscribers to the users mailing list
• 183 subscribers to the dev mailing list
The community is active
• ~863 mails per month on users mailing list in Mar 2011 - May 2012 (+25 %)
• ~167 mails per month on the dev mailing list in Mar 2011 - May 2012 (-15 %)
• Avg. 535 commits per month in Mar 2011 - May 2012 (+5 %)
23. CXF
Simple, flexible, interceptor based, standards compliant
framework for building JAX-WS and JAX-RS services and
consumers
24. Additional - Application development tools
Spring-DM and Spring
Aries Blueprint
Google Guice
Felix iPojo
And so on.....
25. Do I need all of that?
Nope, many solutions will need just a few things
• jaxb, camel, jms, soap, rest and perhaps jdbc
• Cut the container down to fit your needs
• We don’t need to load all of the 100+ Apache Camel components
• Pick and choose!
Should I run that messaging solution inside the “ESB”
• Entirely up to you, let us look a little deeper at that in a sec.
Can I test these solutions or am I stuck with System.out.println
and a remote debugger?
26. I’ve heard that OSGi sucks?
Yes, It is a bit more complex to manage a full lifecycle
• On the other hand you get reliable hot-deployment and things like configuration,
service management and subscriptions out of the box.
Yes, package level import export can be confusing
• So make sure you use packages in a nice modular way.
Yes, it takes a while to learn the tooling
• Then did you wake up one day just knowing that WAR or EAR layout?
Yes, you need to learn the classpath and a bit about
classloading
• Which’ll make you a better developer in the end of the day.
• Many problems in this area actually are created by applying EE classloaders to an
OSGi environment.
27. More suckage
You have to write modular software!!!!
• Yes, you’ll benefit from writing modular software
Heh :)
Yes, some third party libraries are “Fluster-Clucks” in OSGi
• You can always re-package em’ or use something else.
• ServiceMix bundles
Yes, logging is strange and odd
• So Pax logging is provided
Yes, It was hard to get Hibernate working at one point in time
• And there are like a bazillion blogs about getting it working.....
28. W is the simplest way of addressing teh SUKZ?
hat
Use Apache Maven.
• Just accept it.
Designate a maven fascist
• Keep your builds nice and clean
• Physically and monetary make sure that you abuse people breaking builds
• Structure your projects as early as you can
• Don’t be afraid of using a truckload of modules
• Get familiar with the OSGi plugins, let your modules inherit base settings
Go steal that build setup for Apache Karaf or Apache Camel!
Use a whiteboard and perhaps do a few diagrams every now and
then?
Other ways of 100% guaranteed suckcess is to rely only on blogs,
random advice on irc and never reading source code.
29. And keep this in mind...
Spring-DM... it’s a bit like Trump’s hair...
Spring-DM... it’s a bit like Trump’s hair...
Its there... it covers quite a bit...
but it’s kinda hard to recommend.
Looks great... but needs a lot of maintenance
or it becomes a big mess.
30. Let’s say we liked the Apache stack!
And let us say we have a very simple and silly use-case
• We want to accept incoming SOAP requests
• We need to inspect those HTTP headers and then use them to route
• We want this to be dynamic and flexible
We might want to add a more complex routing engine later
• We want this to be modular and possible to run across multiple systems
This tells us we could use
• Spring or Aries Blueprint, Google Guice or plain old Java
• We will use Apache CXF for the WS part
• We will use Apache Camel for connecting this and routing
• We will use JMS via Apache Camel-JMS
32. Proof of concept outline
Let us use some EIP pattern symbols to outline this
• We haven’t really made any technology statements here
We are going to use Apache ActiveMQ because I said so.
It could easily have been any other JMS provider.
• We know we’ll have a synchronous part
Web-services by nature are.
• We’ll try and make all the other parts asynchronous
• We’ll also make sure that the participants don’t need strong integration or visibility
of each other, thus we’ll (hopefully) end up with some pretty re-usable modules
• Another good approach is to also use sequence diagrams
33. Let’s craft some code!
We’ll start with the Maven setup
• Use properties
• Define a parent
• Like said previously, steal a good setup
• Inherit, inherit, inherit
• Use modules
We also ripped a WSDL from Apache CXF
• SOAP Entry point
• Gives something to code-gen against
• Common use-case
36. W else do we want here?
hat
A “Dynamic” Router!
• We’ll craft a simple header based Dynamic router from eip patterns.
Technically it is in between a routing slip with termination and a real control
channel pattern, we utilize JMS as our control channel for this.
We use this example to play with destinations from SoapUI
37. The “Dynamic Router”
Yep, it is just a Java class.
• Imagine how easily you could integrate this with.... Anything!
38. W build that type of a route?
hy
We are able to turn “endpoints”, “routing”, “systems” or whatever
you want to call it into simple destination names.
As we get more complex we only need to control the hop-
building.
We can break all these modules apart
39. So let’s sum up what we have at this point
We defined a CXF (JAX-WS) Endpoint, generated stubs and
tied it into Camel
• The MEP (Message Exchange Pattern) is IN/OUT since we use CXF
We then dropped in a router “Our Dynamic Router”
• It’ll listen to a JMS queue called recognizer
• Then it’ll start inspecting headers for a location
40. Header inspections
A simple processor, if we wanted to we could make similar
transformations in XML, Simple, Scala and so on.
We are just modifying the message header info.
41. W about that testing?
hat
We are going to be dealing with an OSGi environment.
• Like an EE container it makes testing a tad less fun....
Full blown container testing with something like SoapUI
• Can be scripted and automated in Maven
Full OSGi tests with Pax-Exam
• Allows you to wire up a full container
Simple OSGi registry tests with PojoSr
• Let’s you simulate an environment so you can test BluePrint!
Functional Junit/TestNG tests with Camel-Test.
• Should be combined with at least one of the above!
42. Let’s start with a Camel-T T
est est.
Several flavors
• Junit, Junit4, TestNG, Spring and so on.
• Exposes easy access to context, components and inlined route-builders
44. A pax exam based test - Straight from Camel’s source
45. Pax-Exam
Extend the Camel Tests
Will be fairly slow, newer Pax-Exam is improved
Fires up a complete container
Use this in profiles so you can “turn it off”
46. PojoSr ; Extend Camels Blueprint T Support
est
Simple route, pretty much the same syntax as Spring
This is available from Apache Camel 2.10
49. W got slightly distracted in looking at T Options...
e est
First of, let us just do a Camel Test, we want to verify that the
Dynamic routing works as expected.
• As you can see, none of these routes know about each other.
50. Then we want to check that we can load the WS
Blueprint context - We rely on some AMQ Magic as well!
51. And our wiring “Test”
First off, the “Dummy Response”
• We simulate the real route that only grabs a message and dumps it on a queue,
we add 2 things.
Response building
A Mock destination for validation.
52. T method
est
With this we have a full integrations test including
• JMS, CXF, OSGi ServiceRegistry, SPI lookup, Component Discovery, JAXB
validation and schema generation....
53. Green Bars!!!
We’ve tested most functionality, all we really need to do
• Write the “response-builder”
• Make that response builder a bit more dynamic so it can work
Against several queues
With different responses
Yes, it is a dummy class. But since it is a Java Object it is pretty darn easy to let
your imagination run free.
54. Dynamic config data in Blueprint
Works like springs property placeholder
Allows us to set defaults
A persistent-id corresponds to a config file in ./etc in ServiceMix
or Karaf.
55. How we’ll be using this dynamic data
We basically can use this to write completely dynamic steps that
will restart on a configuration change from
• File changes
• Code changes
• JMX changes
56. Proof of concept outline
We now have the following modules
• customer-service : A model library
• customer-service-ws : The webservices endpoint
• dynamic-routing : A generic module for routing
• response-builder : A simple module that returns a payload
• That means that all of the needed patterns are fulfilled.
• Competing consumer is a natural thing in JMS
• Eventing is a natural part of Apache Camel
• Request Reply is “Handled for us” based on endpoint and MEP
57.
58. T sum everything up
o
Simple OSGi build, we added exports in one place.
Completely modular design
Re-usable bundles
We can deploy this on one or many machines
We can “cluster” a network of brokers
We can easily change transport protocols
We are covered by tests
59. Thank you very much for your attention!
Questions?
Links
• http://camel.apache.org/
• http://servicemix.apache.org/
• http://cxf.apache.org/
• http://activemq.apache.org/
• http://karaf.apache.org/