Mais conteúdo relacionado Semelhante a Scaling the Data Distribution Service to Global Networks (7) Mais de Angelo Corsaro (20) Scaling the Data Distribution Service to Global Networks1. Angelo Corsaro, Ph.D.
PrismTech
angelo.corsaro@prismtech.com
Sara Tucci-Piergiovanni, Ph.D.
University of Rome “La Sapienza”
sara.tucci@dis.uniroma1.it
OpenSplice DDS
Delivering Performance, Openness, and Freedom
Scaling the Data Distribution
Service to Global Networks
2. State of the Art
Scaling DDS to Global Networks
Agenda
Extensions to the Standard
Concluding Remarks
3. Addressing Data Distribution Challenges
DDS is standard designed to address the data-distribution challenges across
The OMG DDS Standard a wide class of Defense and Aerospace Applications
‣ Introduced in 2004 to address the Data
Distribution challenges faced by a wide
class of Defense and Aerospace
Applications
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
‣ Key requirement for the standard were its
ability to deliver very high performance while
seamlessly scaling from embedded to ultra-
large-scale deployments
‣ Today recommended by key administration
worldwide and widely adopted across
several different application domains, such
as, Automated Trading, Simulations, SCADA,
Telemetry, etc.
© 2009, PrismTech. All Rights Reserved
4. Addressing Data Distribution Challenges
‣ DDS is swiftly being adopted as the
standard for providing System of Systems
with ubiquitous and access to tactical data
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
‣ DDS deployments on WAN and sizable MAN
have also proven it fitness with
Metropolitan/Wide Area Networks
So... Why I am doing this presentation?
© 2009, PrismTech. All Rights Reserved
5. Challenges
DDS presents today some limitations with respect to addressing large scale geographical
deployments, these are:
Discovery
‣ Standard Discovery Protocol Requires Multicast
‣ Scalability
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
‣ Limited Control over Topics Visibility, e.g., choosing what to expose to the external world
Communication
‣ Standard Communication Protocol Relies on UDP
‣ No Support for Compression
‣ Difficult NAT/Firewall traversal
‣ Sub-Optimal Data Distribution for WAN Scenario (limited exploitation of multicast)
‣ Scalability
© 2009, PrismTech. All Rights Reserved
6. Challenges
Publisher Publisher
B B
Subscriber Subscriber
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
m m
A F A F
Publisher J Publisher
D C J
X Z
K
E
K
Subscriber Y
Subscriber
DDS-1
Publisher
Wan
© 2009, PrismTech. All Rights Reserved
7. Challenges
Publisher A Publisher
B B
Subscriber Subscriber
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
m m
A F A F
Publisher J Publisher
D C J
X Z
K
E
K
Subscriber Y
Subscriber
DDS-1
Publisher
Wan
© 2009, PrismTech. All Rights Reserved
8. State of the Art
Scaling DDS to Global Networks
Agenda
Extensions to the Standard
Concluding Remarks
9. Looking at Internet Apps.
When looking at a solutions for extending DDS applicability to Internet Scale Systems, we can’t
ignore architectures adopted by very successful Internet Applications.
‣ P2P application like KaZaa have shown to scale very well and to support millions of users while
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
effectively managing millions of subjects
‣ Application like Skype are connecting millions of people and managing in real-time they
presence (discovery)
© 2009, PrismTech. All Rights Reserved
10. Architecture for Internet Scale DDS
‣ DDS-Level Routers will be introduced that H 1,1,h
establish a Peer-to-Peer overlay that N2,5 R N1,1 H 1,1,1
relays DDS data over the WAN SN
R
H 1,1,0
‣ The Router provides a single access point for H 2,1,m
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
deciding what to expose to the external world SN
H 1,0,k
‣ The Router can perform Topic Transformation, etc. SN
‣ A Network of Super-Nodes is used for
R
N2,0 H 1,0,1 N1,0
ensuring: H 2,0,i H 1,0,0 R
R N2,4
‣ Scalable (Global) Discovery R SN
‣ Subscription Management (when crossing the N2,1
boundaries of a System) SN H 2,1,n
‣ Support for establishing communication between
Pub and Sub (perhaps via STUNT to circumvent N2,2 R R N2,3
H 2,1,j
NAT/FIREWALL issues)
H 2,1,k H 2,1,h
© 2009, PrismTech. All Rights Reserved
11. DDS Router
‣ DDS Router communicates with the
local DDS Domain by Multicast Ni
R
‣ DDS Router are configured with the
set of topics to be exposed along with
potential transformations
‣ Router2Router communication is
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
R N
carried on by RTPS over TCP (since k
TCP has been tuned over years to
work fine in WAN
‣ If required the Router takes care of
compressing Data
‣ DDS Router Runs a “variation” or
DDSI/RTPS
‣ Topic, Subscription and Publication are propagated to Super Nodes
‣ Information available into super nodes ensure that local reader/writer will be properly matched
© 2009, PrismTech. All Rights Reserved
12. DDS Super-Nodes (SN)
‣ Super-Nodes store all the “discovery information”
‣ List of Topics available in the system R
SN
‣ Associations between Topics and Router role (e.g. Pub or SN
R
Sub)
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
‣ A distributed protocol is used to replicated data,
SN
eventually, on all replicas
SN
‣ Router keep a list of well-known Super-Nodes to
bootstrap discovery R
R
‣ NOTE: Super-Nodes can also be used to support
the implementation of STUNT-like protocols to
facilitate NAT/Firewall trespassing
© 2009, PrismTech. All Rights Reserved
13. How it Works
{P(A),S(B),P(C)} {S(B), S(D),P(E)}
DDS R1 SN1 SN4 R4 DDS
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
SN2 SN3
{S(A),P(B),P(C)} R2 R3 {S(A),P(D),S(E)}
DDS DDS
© 2009, PrismTech. All Rights Reserved
14. How it Works
{P(A),S(B),P(C)} R1-{P(A),S(B),P(C)} {S(B), S(D),P(E)}
DDS R1 SN1 SN4 R4 DDS
R4-{S(B), S(D),P(E)}
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
SN2 SN3 R3-{S(A),P(D),S(E)}
R2-{S(A),P(B),P(C)}
{S(A),P(B),P(C)} R2 R3 {S(A),P(D),S(E)}
DDS DDS
© 2009, PrismTech. All Rights Reserved
15. How it Works
{P(A),S(B),P(C)} {S(B), S(D),P(E)}
DDS R1 SN1 SN4 R4 DDS
R1-{P(A),S(B),P(C)}
R2-{S(A),P(B),P(C)}
R3-{S(A),P(D),P(E)}
R4-{S(B), S(D),S(E)}
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
SN2 SN3
{S(A),P(B),P(C)} R2 R3 {S(A),P(D),S(E)}
DDS DDS
© 2009, PrismTech. All Rights Reserved
16. How it Works
{P(A),S(B),P(C)} {S(B), S(D),P(E)}
DDS R1 SN1 SN4 R4 DDS
{P(A)} R1-{P(A),S(B),P(C)}
R2-{S(A),P(B),P(C)}
{P(A)} R3-{S(A),P(D),P(E)} {P(E)}
R4-{S(B), S(D),S(E)}
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
SN2 SN3
{P(B)} {P(D)}
{S(A),P(B),P(C)} {P(B)} R3 {S(A),P(D),S(E)}
R2
DDS DDS
© 2009, PrismTech. All Rights Reserved
17. How it Works
{P(A),S(B),P(C)} {S(B), S(D),P(E)}
DDS A R1 SN1 SN4 R4 DDS
{P(A)} R1-{P(A),S(B),P(C)}
R2-{S(A),P(B),P(C)}
{P(A)} R3-{S(A),P(D),P(E)} {P(E)}
R4-{S(B), S(D),S(E)}
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
SN2 SN3
{P(B)} {P(D)}
{S(A),P(B),P(C)} {P(B)} R3 {S(A),P(D),S(E)}
R2
A A
A
DDS DDS
© 2009, PrismTech. All Rights Reserved
18. State of the Art
Scaling DDS to Global Networks
Agenda
Extensions to the Standard
Concluding Remarks
19. Extension to the DDS Standard
‣ What described in the previous slides can easily be implemented in an
interoperable manner, exploiting the pre-designed extension points present in
the DDS Standard
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
‣ Areas of extension include:
‣Discovery
‣DDSI/RTPS
© 2009, PrismTech. All Rights Reserved
20. Discovery Extension
1‣ A new Discovery Protocol should be R
added that allows to discover 1 SN
SN
relevant information (Topics, R
2
Subscriptions, Publications) via the
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
Super-Nodes SN
SN
2‣ A new protocol to be used between
Super-Nodes should also be R
standardized!
R
© 2009, PrismTech. All Rights Reserved
21. DDSI/RTPS Protocol xTensions
DDSI/RTPS Should be extended DDS R1 SN1 SN4 R4 DDS
to include:
SN2 SN3
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
‣ Support for TCP/IP
‣Offloading reliability from DDSI to TCP R2 R3
whenever possible DDS DDS
‣ Compression
© 2009, PrismTech. All Rights Reserved
22. State of the Art
Scaling DDS to Global Networks
Agenda
Extensions to the Standard
Concluding Remarks
23. Concluding Remarks
‣ The DDS Standard includes today some known
limitation when trying to address Ultra-Large-
Scale Systems
‣ These limitation mostly impact its scalability and
efficiency
Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
‣ The good news is that exploiting available
extension points in the standard it is possible to
extend the DDS Architectural Style to scale to
Ultra-Large-Scale Systems
‣ The Router/Super-Nodes approach builds over
years of experience in the domain of Internet
Scale applications, such as, KaZaa, Skype, etc.
‣ Thus, PrismTech will work toward update of the
DDS Spec to include these extension points.
© 2009, PrismTech. All Rights Reserved