Distributed Programming Abstractions are discussed using Web Services, Grids, Parallel Computing and Multicore architectures

1. Linking Programming models between Grids, Web 2.0 and Multicore Distributed Programming Abstractions Workshop NESC Edinburgh UK May 31 2007 Geoffrey Fox Computer Science, Informatics, Physics Pervasive Technology Laboratories Indiana University Bloomington IN 47401 [email_address] http:// www.infomall.org

10. What is …? What if …? Is it …? R ecognition M ining S ynthesis Create a model instance RMS: Recognition Mining Synthesis Model-based multimodal recognition Find a model instance Model Real-time analytics on dynamic, unstructured, multimodal datasets Photo-realism and physics-based animation Model-less Real-time streaming and transactions on static – structured datasets Very limited realism Intel has probably most sophisticated analysis of future “killer” multicore applications – they are “just” standard Grid and parallel computing Tomorrow Today

11. What is a tumor? Is there a tumor here? What if the tumor progresses? It is all about dealing efficiently with complex multimodal datasets R ecognition M ining S ynthesis Images courtesy: http://splweb.bwh.harvard.edu:8000/pages/images_movies.html

12. Intel’s Application Stack PC07Intro [email_address]

35. APIs/Mashups per Protocol Distribution Number of Mashups Number of APIs REST SOAP XML-RPC REST, XML-RPC REST, XML-RPC, SOAP REST, SOAP JS Other google maps netvibes live.com virtual earth google search amazon S3 amazon ECS flickr ebay youtube 411sync del.icio.us yahoo! search yahoo! geocoding technorati yahoo! images trynt yahoo! local

40. The Ten areas covered by the 60 core WS-* Specifications WSRP (Remote Portlets) 10: Portals and User Interfaces WS-Policy, WS-Agreement 9: Policy and Agreements WSDM, WS-Management, WS-Transfer 8: Management WSRF, WS-MetadataExchange, WS-Context 7: System Metadata and State UDDI, WS-Discovery 6: Service Discovery WS-Security, WS-Trust, WS-Federation, SAML, WS-SecureConversation 5: Security BPEL, WS-Choreography, WS-Coordination 4: Workflow and Transactions WS-Notification, WS-Eventing (Publish-Subscribe) 3: Notification WS-Addressing, WS-MessageDelivery; Reliable Messaging WSRM; Efficient Messaging MOTM 2: Service Internet XML, WSDL, SOAP 1: Core Service Model Typical Grid/Web Service Examples WS-* Specification Area

41. WS-* Areas and Web 2.0 Start Pages, AJAX and Widgets(Netvibes) Gadgets 10: Portals and User Interfaces Service dependent. Processed by application 9: Policy and Agreements WS-Transfer style Protocols GET PUT etc. 8: Management==Interaction Processed by application – no system state – Microformats are a universal metadata approach 7: System Metadata and State http://www.programmableweb.com 6: Service Discovery SSL, HTTP Authentication/Authorization, OpenID is Web 2.0 Single Sign on 5: Security Mashups, Google MapReduce Scripting with PHP JavaScript …. 4: Workflow and Transactions (no Transactions in Web 2.0) Hard with HTTP without polling – JMS perhaps? 3: Notification No special QoS. Use JMS or equivalent? 2: Service Internet XML becomes optional but still useful SOAP becomes JSON RSS ATOM WSDL becomes REST with API as GET PUT etc. Axis becomes XmlHttpRequest 1: Core Service Model Web 2.0 Approach WS-* Specification Area

42. WS-* Areas and Multicore Web 2.0 technology popular 10: Portals and User Interfaces Handled by application 9: Policy and Agreements Interaction between objects key issue in parallel programming trading off efficiency versus performance 8: Management == Interaction Environment Variables 7: System Metadata and State Use libraries 6: Service Discovery Not so important intrachip 5: Security Many approaches; scripting languages popular 4: Workflow and Transactions Publish-Subscribe for events and Interrupts 3: Notification Not so important intrachip 2: Service Internet Fine grain Java C# C++ Objects and coarse grain services as in DSS . Information passed explicitly or by handles. MPI needs to be updated to handle non scientific applications as in CCR 1: Core Service Model Typical Grid/Web Service Examples WS-* Specification Area

45. Overhead (latency) of AMD 4-core PC with 4 execution threads on MPI style Rendezvous Messaging for Shift and Exchange implemented either as two shifts or as custom CCR pattern. Compute time is 10 seconds divided by number of stages Stages (millions) Time Microseconds Rendezvous exchange as two shifts Rendezvous exchange customized for MPI Rendezvous Shift

46. Overhead (latency) of INTEL 8-core PC with 8 execution threads on MPI style Rendezvous Messaging for Shift and Exchange implemented either as two shifts or as custom CCR pattern. Compute time is 15 seconds divided by number of stages Stages (millions) Time Microseconds Rendezvous exchange as two shifts Rendezvous exchange customized for MPI Rendezvous Shift