SlideShare a Scribd company logo
1 of 21
Inter-Process/Task Communication With Message Queues William McVey < [email_address] > PyOhio July 26, 2009
Intro ,[object Object],[object Object],[object Object]
Description of the Problem ,[object Object],[object Object],[object Object],[object Object]
Architecture
Key Difficulties ,[object Object],[object Object],[object Object],[object Object]
Lesson learned Do  not  screw with Apache's process model.
Rediscovering Queues ,[object Object],[object Object],[object Object],[object Object],[object Object]
Generic message broker ,[object Object],[object Object],[object Object],[object Object],[object Object]
beanstalkd/beanstalkc ,[object Object],[object Object],[object Object]
The need for something more ,[object Object],[object Object],[object Object],[object Object]
memcacheq ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
AMQP ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
AMQP
AMQP Message Routing Image from:  Messaging Tutorial - AMQP Programming Tutorial for C++, Java, Python, and C# Copyright © 2008 Red Hat, Inc. Under the Open Publication License
ØMQ -  http://zeromq.org/ ,[object Object],[object Object],[object Object],[object Object],[object Object]
RabbitMQ ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
amqplib / carrot py-amqplib  is a client library around the AMQP protocol.Fairly low level for my needs though, so a little digging found  carrot
carrot sample >>>   from   carrot.messaging   import  Publisher, Consumer >>>   class   PostOfficePublisher (Publisher): ...   exchange  =   &quot;sorting_room&quot; ...   routing_key  =   &quot;jason&quot; >>>   class   PostOfficeConsumer (Consumer): ...   queue  =   &quot;po_box&quot; ...   exchange  =   &quot;sorting_room&quot; ...   routing_key  =   &quot;jason&quot; ... ...   def   receive ( self , message_data, message): ...   &quot;&quot;&quot;Called when we receive a message.&quot;&quot;&quot; ...   print ( &quot;Received:  %s &quot;   %  message_data)
carrot sample >>>   from   ConfigParser   import  ConfigParser >>>  config  =  ConfigParser() >>>  config . read( &quot;application.ini&quot; ) >>>   from   carrot.connection   import  AMQPConnection >>>  amqpconn  =  AMQPConnection( ...   hostname  =  config . get( &quot;broker&quot; ,  &quot;host&quot; ), ...   port  =  config . get( &quot;broker&quot; ,  &quot;port&quot; ), ...   userid  =  config . get( &quot;broker&quot; ,  &quot;userid&quot; ), ...   password  =  config . get( &quot;broker&quot; ,  &quot;password&quot; ), ...   vhost  =  config . get( &quot;broker&quot; ,  &quot;vhost&quot; )) >>>  PostOfficePublisher(connection = amqpconn) . send( ...   { &quot;My message&quot; : [ &quot;foo&quot; ,  &quot;bar&quot; ,  &quot;baz&quot; ]}) >>>  PostOfficeConsumer(connection = amqpconn) . next() Received: { &quot;My message&quot; : [ &quot;foo&quot; ,  &quot;bar&quot; ,  &quot;baz&quot; ]}
multiprocessing ,[object Object],[object Object],[object Object],[object Object],[object Object]
In Summary ,[object Object],[object Object],[object Object],[object Object],[object Object]

More Related Content

What's hot

Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring frameworkAssurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring frameworkGosuke Miyashita
 
XMPP Technical Overview + Jingle Protocol Study
XMPP Technical Overview + Jingle Protocol StudyXMPP Technical Overview + Jingle Protocol Study
XMPP Technical Overview + Jingle Protocol StudyValérian Saliou
 
NetScaler Web2.0 Push Technology Overview
NetScaler Web2.0 Push Technology OverviewNetScaler Web2.0 Push Technology Overview
NetScaler Web2.0 Push Technology Overviewkvamsi
 
How Danga::Socket handles asynchronous processing and how to write asynchrono...
How Danga::Socket handles asynchronous processing and how to write asynchrono...How Danga::Socket handles asynchronous processing and how to write asynchrono...
How Danga::Socket handles asynchronous processing and how to write asynchrono...Gosuke Miyashita
 
Communication in Python and the C10k problem
Communication in Python and the C10k problemCommunication in Python and the C10k problem
Communication in Python and the C10k problemJose Galarza
 
Expand Your Testing with Virtual Services
Expand Your Testing with Virtual ServicesExpand Your Testing with Virtual Services
Expand Your Testing with Virtual ServicesAmber Race
 
LCA2014 - Introduction to Go
LCA2014 - Introduction to GoLCA2014 - Introduction to Go
LCA2014 - Introduction to Godreamwidth
 
Real-Time Web applications with WebSockets
Real-Time Web applications with WebSocketsReal-Time Web applications with WebSockets
Real-Time Web applications with WebSocketsStanislav Zozulia
 
What To Learn First in PHP
What To Learn First in PHPWhat To Learn First in PHP
What To Learn First in PHPTessa Mero
 
Ruby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.com
Ruby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.comRuby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.com
Ruby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.comIlya Grigorik
 
Debugging over tcp and http
Debugging over tcp and httpDebugging over tcp and http
Debugging over tcp and httpKaniska Mandal
 
parenscript-tutorial
parenscript-tutorialparenscript-tutorial
parenscript-tutorialtutorialsruby
 
Augmenting Web Services with SMS and XMPP
Augmenting Web Services with SMS and XMPPAugmenting Web Services with SMS and XMPP
Augmenting Web Services with SMS and XMPPSam Keen
 
Locker: distributed consistent locking
Locker: distributed consistent lockingLocker: distributed consistent locking
Locker: distributed consistent lockingKnut Nesheim
 
Smart Gamma - Real-Time Web applications with PHP and Websocket.
Smart Gamma - Real-Time Web applications with PHP and Websocket.Smart Gamma - Real-Time Web applications with PHP and Websocket.
Smart Gamma - Real-Time Web applications with PHP and Websocket.Evgeniy Kuzmin
 

What's hot (20)

Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring frameworkAssurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
 
XMPP Technical Overview + Jingle Protocol Study
XMPP Technical Overview + Jingle Protocol StudyXMPP Technical Overview + Jingle Protocol Study
XMPP Technical Overview + Jingle Protocol Study
 
Perl comet
Perl cometPerl comet
Perl comet
 
NetScaler Web2.0 Push Technology Overview
NetScaler Web2.0 Push Technology OverviewNetScaler Web2.0 Push Technology Overview
NetScaler Web2.0 Push Technology Overview
 
How Danga::Socket handles asynchronous processing and how to write asynchrono...
How Danga::Socket handles asynchronous processing and how to write asynchrono...How Danga::Socket handles asynchronous processing and how to write asynchrono...
How Danga::Socket handles asynchronous processing and how to write asynchrono...
 
Communication in Python and the C10k problem
Communication in Python and the C10k problemCommunication in Python and the C10k problem
Communication in Python and the C10k problem
 
Expand Your Testing with Virtual Services
Expand Your Testing with Virtual ServicesExpand Your Testing with Virtual Services
Expand Your Testing with Virtual Services
 
Cramp websockets
Cramp websocketsCramp websockets
Cramp websockets
 
LCA2014 - Introduction to Go
LCA2014 - Introduction to GoLCA2014 - Introduction to Go
LCA2014 - Introduction to Go
 
Real-Time Web applications with WebSockets
Real-Time Web applications with WebSocketsReal-Time Web applications with WebSockets
Real-Time Web applications with WebSockets
 
What To Learn First in PHP
What To Learn First in PHPWhat To Learn First in PHP
What To Learn First in PHP
 
Ruby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.com
Ruby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.comRuby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.com
Ruby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.com
 
Debugging over tcp and http
Debugging over tcp and httpDebugging over tcp and http
Debugging over tcp and http
 
parenscript-tutorial
parenscript-tutorialparenscript-tutorial
parenscript-tutorial
 
Php Debugger
Php DebuggerPhp Debugger
Php Debugger
 
Augmenting Web Services with SMS and XMPP
Augmenting Web Services with SMS and XMPPAugmenting Web Services with SMS and XMPP
Augmenting Web Services with SMS and XMPP
 
Gearman and Perl
Gearman and PerlGearman and Perl
Gearman and Perl
 
A few words about WAMP
A few words about WAMPA few words about WAMP
A few words about WAMP
 
Locker: distributed consistent locking
Locker: distributed consistent lockingLocker: distributed consistent locking
Locker: distributed consistent locking
 
Smart Gamma - Real-Time Web applications with PHP and Websocket.
Smart Gamma - Real-Time Web applications with PHP and Websocket.Smart Gamma - Real-Time Web applications with PHP and Websocket.
Smart Gamma - Real-Time Web applications with PHP and Websocket.
 

Viewers also liked

DDS Interoperability Demo 2013 (Washington DC)
DDS Interoperability Demo 2013 (Washington DC)DDS Interoperability Demo 2013 (Washington DC)
DDS Interoperability Demo 2013 (Washington DC)Gerardo Pardo-Castellote
 
IPC (Inter-Process Communication) with Shared Memory
IPC (Inter-Process Communication) with Shared MemoryIPC (Inter-Process Communication) with Shared Memory
IPC (Inter-Process Communication) with Shared MemoryHEM DUTT
 
OpenSSL programming (still somewhat initial version)
OpenSSL programming (still somewhat initial version)OpenSSL programming (still somewhat initial version)
OpenSSL programming (still somewhat initial version)Shteryana Shopova
 
Cloud architectural patterns and Microsoft Azure tools
Cloud architectural patterns and Microsoft Azure toolsCloud architectural patterns and Microsoft Azure tools
Cloud architectural patterns and Microsoft Azure toolsPushkar Chivate
 
Distributed messaging with AMQP
Distributed messaging with AMQPDistributed messaging with AMQP
Distributed messaging with AMQPWee Keat Chin
 
OWASP Ireland June Chapter Meeting - Paul Mooney on ARMOR & CSRF
OWASP Ireland June Chapter Meeting - Paul Mooney on ARMOR & CSRFOWASP Ireland June Chapter Meeting - Paul Mooney on ARMOR & CSRF
OWASP Ireland June Chapter Meeting - Paul Mooney on ARMOR & CSRFPaul Mooney
 
Docker - container and lightweight virtualization
Docker - container and lightweight virtualization Docker - container and lightweight virtualization
Docker - container and lightweight virtualization Sim Janghoon
 
Crypto With OpenSSL
Crypto With OpenSSLCrypto With OpenSSL
Crypto With OpenSSLZhi Guan
 
Interprocess Communication
Interprocess CommunicationInterprocess Communication
Interprocess CommunicationDeepak H L
 
The Future of Messaging: RabbitMQ and AMQP
The Future of Messaging: RabbitMQ and AMQP The Future of Messaging: RabbitMQ and AMQP
The Future of Messaging: RabbitMQ and AMQP Eberhard Wolff
 
Inter process communication
Inter process communicationInter process communication
Inter process communicationRJ Mehul Gadhiya
 
Interprocess communication
Interprocess communicationInterprocess communication
Interprocess communicationSushil Singh
 
Microservice vs. Monolithic Architecture
Microservice vs. Monolithic ArchitectureMicroservice vs. Monolithic Architecture
Microservice vs. Monolithic ArchitecturePaul Mooney
 
Architecture | The Future of Messaging: RabbitMQ and AMQP | Eberhard Wolff
Architecture | The Future of Messaging: RabbitMQ and AMQP | Eberhard WolffArchitecture | The Future of Messaging: RabbitMQ and AMQP | Eberhard Wolff
Architecture | The Future of Messaging: RabbitMQ and AMQP | Eberhard WolffJAX London
 
Scaling Apache Storm - Strata + Hadoop World 2014
Scaling Apache Storm - Strata + Hadoop World 2014Scaling Apache Storm - Strata + Hadoop World 2014
Scaling Apache Storm - Strata + Hadoop World 2014P. Taylor Goetz
 
Storm: distributed and fault-tolerant realtime computation
Storm: distributed and fault-tolerant realtime computationStorm: distributed and fault-tolerant realtime computation
Storm: distributed and fault-tolerant realtime computationnathanmarz
 
Realtime Analytics with Storm and Hadoop
Realtime Analytics with Storm and HadoopRealtime Analytics with Storm and Hadoop
Realtime Analytics with Storm and HadoopDataWorks Summit
 

Viewers also liked (20)

DDS Interoperability Demo 2013 (Washington DC)
DDS Interoperability Demo 2013 (Washington DC)DDS Interoperability Demo 2013 (Washington DC)
DDS Interoperability Demo 2013 (Washington DC)
 
IPC (Inter-Process Communication) with Shared Memory
IPC (Inter-Process Communication) with Shared MemoryIPC (Inter-Process Communication) with Shared Memory
IPC (Inter-Process Communication) with Shared Memory
 
Inter Process Communication
Inter Process CommunicationInter Process Communication
Inter Process Communication
 
OpenSSL programming (still somewhat initial version)
OpenSSL programming (still somewhat initial version)OpenSSL programming (still somewhat initial version)
OpenSSL programming (still somewhat initial version)
 
Cloud architectural patterns and Microsoft Azure tools
Cloud architectural patterns and Microsoft Azure toolsCloud architectural patterns and Microsoft Azure tools
Cloud architectural patterns and Microsoft Azure tools
 
Distributed messaging with AMQP
Distributed messaging with AMQPDistributed messaging with AMQP
Distributed messaging with AMQP
 
OWASP Ireland June Chapter Meeting - Paul Mooney on ARMOR & CSRF
OWASP Ireland June Chapter Meeting - Paul Mooney on ARMOR & CSRFOWASP Ireland June Chapter Meeting - Paul Mooney on ARMOR & CSRF
OWASP Ireland June Chapter Meeting - Paul Mooney on ARMOR & CSRF
 
Docker - container and lightweight virtualization
Docker - container and lightweight virtualization Docker - container and lightweight virtualization
Docker - container and lightweight virtualization
 
Crypto With OpenSSL
Crypto With OpenSSLCrypto With OpenSSL
Crypto With OpenSSL
 
Interprocess Communication
Interprocess CommunicationInterprocess Communication
Interprocess Communication
 
The Future of Messaging: RabbitMQ and AMQP
The Future of Messaging: RabbitMQ and AMQP The Future of Messaging: RabbitMQ and AMQP
The Future of Messaging: RabbitMQ and AMQP
 
Inter process communication
Inter process communicationInter process communication
Inter process communication
 
Interprocess communication
Interprocess communicationInterprocess communication
Interprocess communication
 
Microservice vs. Monolithic Architecture
Microservice vs. Monolithic ArchitectureMicroservice vs. Monolithic Architecture
Microservice vs. Monolithic Architecture
 
Architecture | The Future of Messaging: RabbitMQ and AMQP | Eberhard Wolff
Architecture | The Future of Messaging: RabbitMQ and AMQP | Eberhard WolffArchitecture | The Future of Messaging: RabbitMQ and AMQP | Eberhard Wolff
Architecture | The Future of Messaging: RabbitMQ and AMQP | Eberhard Wolff
 
Resource Aware Scheduling in Apache Storm
Resource Aware Scheduling in Apache StormResource Aware Scheduling in Apache Storm
Resource Aware Scheduling in Apache Storm
 
Scaling Apache Storm - Strata + Hadoop World 2014
Scaling Apache Storm - Strata + Hadoop World 2014Scaling Apache Storm - Strata + Hadoop World 2014
Scaling Apache Storm - Strata + Hadoop World 2014
 
Storm: distributed and fault-tolerant realtime computation
Storm: distributed and fault-tolerant realtime computationStorm: distributed and fault-tolerant realtime computation
Storm: distributed and fault-tolerant realtime computation
 
Realtime Analytics with Storm and Hadoop
Realtime Analytics with Storm and HadoopRealtime Analytics with Storm and Hadoop
Realtime Analytics with Storm and Hadoop
 
Yahoo compares Storm and Spark
Yahoo compares Storm and SparkYahoo compares Storm and Spark
Yahoo compares Storm and Spark
 

Similar to Inter-Process/Task Communication With Message Queues

Puppet Camp DC 2015: Distributed OpenSCAP Compliance Validation with MCollective
Puppet Camp DC 2015: Distributed OpenSCAP Compliance Validation with MCollectivePuppet Camp DC 2015: Distributed OpenSCAP Compliance Validation with MCollective
Puppet Camp DC 2015: Distributed OpenSCAP Compliance Validation with MCollectivePuppet
 
Enterprise Messaging with Apache ActiveMQ
Enterprise Messaging with Apache ActiveMQEnterprise Messaging with Apache ActiveMQ
Enterprise Messaging with Apache ActiveMQelliando dias
 
Service Oriented Integration With ServiceMix
Service Oriented Integration With ServiceMixService Oriented Integration With ServiceMix
Service Oriented Integration With ServiceMixBruce Snyder
 
quickguide-einnovator-3-rabbitmq
quickguide-einnovator-3-rabbitmqquickguide-einnovator-3-rabbitmq
quickguide-einnovator-3-rabbitmqjorgesimao71
 
Hector v2: The Second Version of the Popular High-Level Java Client for Apach...
Hector v2: The Second Version of the Popular High-Level Java Client for Apach...Hector v2: The Second Version of the Popular High-Level Java Client for Apach...
Hector v2: The Second Version of the Popular High-Level Java Client for Apach...zznate
 
The Art of Message Queues - TEKX
The Art of Message Queues - TEKXThe Art of Message Queues - TEKX
The Art of Message Queues - TEKXMike Willbanks
 
Hands on with CoAP and Californium
Hands on with CoAP and CaliforniumHands on with CoAP and Californium
Hands on with CoAP and CaliforniumJulien Vermillard
 
JMS Introduction
JMS IntroductionJMS Introduction
JMS IntroductionAlex Su
 
AMIMOTO: WordPress + Amazon Web Services Hands-on PARIS
AMIMOTO: WordPress + Amazon Web Services Hands-on PARISAMIMOTO: WordPress + Amazon Web Services Hands-on PARIS
AMIMOTO: WordPress + Amazon Web Services Hands-on PARISKel
 
What we learnt at carousell tw for golang gathering #31
What we learnt at carousell tw for golang gathering #31What we learnt at carousell tw for golang gathering #31
What we learnt at carousell tw for golang gathering #31Ronald Hsu
 
Introducing TR-069 - An Axiros Workshop for the TR-069 Protocol - Part 1
Introducing TR-069 - An Axiros Workshop for the TR-069 Protocol - Part 1Introducing TR-069 - An Axiros Workshop for the TR-069 Protocol - Part 1
Introducing TR-069 - An Axiros Workshop for the TR-069 Protocol - Part 1Axiros
 
Introduction to Apache ActiveMQ Artemis
Introduction to Apache ActiveMQ ArtemisIntroduction to Apache ActiveMQ Artemis
Introduction to Apache ActiveMQ ArtemisYoshimasa Tanabe
 
Managing and Scaling Puppet - PuppetConf 2014
Managing and Scaling Puppet - PuppetConf 2014Managing and Scaling Puppet - PuppetConf 2014
Managing and Scaling Puppet - PuppetConf 2014Miguel Zuniga
 
Managing and Scaling Puppet - PuppetConf 2014
Managing and Scaling Puppet - PuppetConf 2014Managing and Scaling Puppet - PuppetConf 2014
Managing and Scaling Puppet - PuppetConf 2014Puppet
 
Fast SOA with Apache Synapse
Fast SOA with Apache SynapseFast SOA with Apache Synapse
Fast SOA with Apache SynapsePaul Fremantle
 
Implementing WebServices with Camel and CXF in ServiceMix
Implementing WebServices with Camel and CXF in ServiceMixImplementing WebServices with Camel and CXF in ServiceMix
Implementing WebServices with Camel and CXF in ServiceMixAdrian Trenaman
 
AWS Lambda from the trenches
AWS Lambda from the trenchesAWS Lambda from the trenches
AWS Lambda from the trenchesYan Cui
 

Similar to Inter-Process/Task Communication With Message Queues (20)

Puppet Camp DC 2015: Distributed OpenSCAP Compliance Validation with MCollective
Puppet Camp DC 2015: Distributed OpenSCAP Compliance Validation with MCollectivePuppet Camp DC 2015: Distributed OpenSCAP Compliance Validation with MCollective
Puppet Camp DC 2015: Distributed OpenSCAP Compliance Validation with MCollective
 
Enterprise Messaging with Apache ActiveMQ
Enterprise Messaging with Apache ActiveMQEnterprise Messaging with Apache ActiveMQ
Enterprise Messaging with Apache ActiveMQ
 
Service Oriented Integration With ServiceMix
Service Oriented Integration With ServiceMixService Oriented Integration With ServiceMix
Service Oriented Integration With ServiceMix
 
quickguide-einnovator-3-rabbitmq
quickguide-einnovator-3-rabbitmqquickguide-einnovator-3-rabbitmq
quickguide-einnovator-3-rabbitmq
 
Tbp
TbpTbp
Tbp
 
Hector v2: The Second Version of the Popular High-Level Java Client for Apach...
Hector v2: The Second Version of the Popular High-Level Java Client for Apach...Hector v2: The Second Version of the Popular High-Level Java Client for Apach...
Hector v2: The Second Version of the Popular High-Level Java Client for Apach...
 
The Art of Message Queues - TEKX
The Art of Message Queues - TEKXThe Art of Message Queues - TEKX
The Art of Message Queues - TEKX
 
Hands on with CoAP and Californium
Hands on with CoAP and CaliforniumHands on with CoAP and Californium
Hands on with CoAP and Californium
 
JMS Introduction
JMS IntroductionJMS Introduction
JMS Introduction
 
AMIMOTO: WordPress + Amazon Web Services Hands-on PARIS
AMIMOTO: WordPress + Amazon Web Services Hands-on PARISAMIMOTO: WordPress + Amazon Web Services Hands-on PARIS
AMIMOTO: WordPress + Amazon Web Services Hands-on PARIS
 
What we learnt at carousell tw for golang gathering #31
What we learnt at carousell tw for golang gathering #31What we learnt at carousell tw for golang gathering #31
What we learnt at carousell tw for golang gathering #31
 
Tomcat 6: Evolving our server
Tomcat 6: Evolving our serverTomcat 6: Evolving our server
Tomcat 6: Evolving our server
 
Introducing TR-069 - An Axiros Workshop for the TR-069 Protocol - Part 1
Introducing TR-069 - An Axiros Workshop for the TR-069 Protocol - Part 1Introducing TR-069 - An Axiros Workshop for the TR-069 Protocol - Part 1
Introducing TR-069 - An Axiros Workshop for the TR-069 Protocol - Part 1
 
2 Asp Dot Net Ajax Extensions
2 Asp Dot Net Ajax Extensions2 Asp Dot Net Ajax Extensions
2 Asp Dot Net Ajax Extensions
 
Introduction to Apache ActiveMQ Artemis
Introduction to Apache ActiveMQ ArtemisIntroduction to Apache ActiveMQ Artemis
Introduction to Apache ActiveMQ Artemis
 
Managing and Scaling Puppet - PuppetConf 2014
Managing and Scaling Puppet - PuppetConf 2014Managing and Scaling Puppet - PuppetConf 2014
Managing and Scaling Puppet - PuppetConf 2014
 
Managing and Scaling Puppet - PuppetConf 2014
Managing and Scaling Puppet - PuppetConf 2014Managing and Scaling Puppet - PuppetConf 2014
Managing and Scaling Puppet - PuppetConf 2014
 
Fast SOA with Apache Synapse
Fast SOA with Apache SynapseFast SOA with Apache Synapse
Fast SOA with Apache Synapse
 
Implementing WebServices with Camel and CXF in ServiceMix
Implementing WebServices with Camel and CXF in ServiceMixImplementing WebServices with Camel and CXF in ServiceMix
Implementing WebServices with Camel and CXF in ServiceMix
 
AWS Lambda from the trenches
AWS Lambda from the trenchesAWS Lambda from the trenches
AWS Lambda from the trenches
 

Recently uploaded

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel Araújo
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slidespraypatel2
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Drew Madelung
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsJoaquim Jorge
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxKatpro Technologies
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Enterprise Knowledge
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024The Digital Insurer
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfEnterprise Knowledge
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityPrincipled Technologies
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processorsdebabhi2
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Scriptwesley chun
 

Recently uploaded (20)

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slides
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 

Inter-Process/Task Communication With Message Queues

  • 1. Inter-Process/Task Communication With Message Queues William McVey < [email_address] > PyOhio July 26, 2009
  • 2.
  • 3.
  • 5.
  • 6. Lesson learned Do not screw with Apache's process model.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
  • 13. AMQP
  • 14. AMQP Message Routing Image from: Messaging Tutorial - AMQP Programming Tutorial for C++, Java, Python, and C# Copyright © 2008 Red Hat, Inc. Under the Open Publication License
  • 15.
  • 16.
  • 17. amqplib / carrot py-amqplib is a client library around the AMQP protocol.Fairly low level for my needs though, so a little digging found carrot
  • 18. carrot sample >>> from carrot.messaging import Publisher, Consumer >>> class PostOfficePublisher (Publisher): ... exchange = &quot;sorting_room&quot; ... routing_key = &quot;jason&quot; >>> class PostOfficeConsumer (Consumer): ... queue = &quot;po_box&quot; ... exchange = &quot;sorting_room&quot; ... routing_key = &quot;jason&quot; ... ... def receive ( self , message_data, message): ... &quot;&quot;&quot;Called when we receive a message.&quot;&quot;&quot; ... print ( &quot;Received: %s &quot; % message_data)
  • 19. carrot sample >>> from ConfigParser import ConfigParser >>> config = ConfigParser() >>> config . read( &quot;application.ini&quot; ) >>> from carrot.connection import AMQPConnection >>> amqpconn = AMQPConnection( ... hostname = config . get( &quot;broker&quot; , &quot;host&quot; ), ... port = config . get( &quot;broker&quot; , &quot;port&quot; ), ... userid = config . get( &quot;broker&quot; , &quot;userid&quot; ), ... password = config . get( &quot;broker&quot; , &quot;password&quot; ), ... vhost = config . get( &quot;broker&quot; , &quot;vhost&quot; )) >>> PostOfficePublisher(connection = amqpconn) . send( ... { &quot;My message&quot; : [ &quot;foo&quot; , &quot;bar&quot; , &quot;baz&quot; ]}) >>> PostOfficeConsumer(connection = amqpconn) . next() Received: { &quot;My message&quot; : [ &quot;foo&quot; , &quot;bar&quot; , &quot;baz&quot; ]}
  • 20.
  • 21.