Querying Data Providing Web Services Manivasakan Sabesan Department of Information Technology Uppsala University Sweden. Abstract Web services are often used for search computing where data is retrieved from servers providing information of different kinds. Such data providing web services return a set of objects for a given set of parameters without any side effects. There is need to enable general and scalable search capabilities of data from data providing web services, which is the topic of this Thesis. The Web Service MEDiator (WSMED) system automatically provides relational views of any data providing web service operations by reading the WSDL documents describing them. These views can be queried with SQL. Without any knowledge of the costs of executing specific web service operations the WSMED query processor automatically and adaptively finds an optimized parallel execution plan calling queried data providing web services. For scalable execution of queries to data providing web services, an algebra operator PAP adaptively parallelizes calls in execution plans to web service operations until no significant performance improvement is measured, based on monitoring the flow from web service operations without any cost knowledge or extensive memory usage. To comply with the Everything as a Service (XaaS) paradigm WSMED itself is implemented as a web service that provides web service operations to query and combine data from data providing web services. A web based demonstration of the WSMED web service provides general SQL queries to any data providing web service operations from a browser. WSMED assumes that all queried data sources are available as web services. To make any data providing system into a data providing web service WSMED includes a subsystem, the web service generator, which generates and deploys the web service operations to access a data source. The WSMED web service itself is generated by the web service generator.

1. Querying Data Providing Web Services Manivasakan Sabesan Uppsala DataBase Laboratory Dept. of Information Technology Uppsala University Sweden

4. Search information Search information through web services Automatically generated parallel programs WSMED US States information Place Details Weather Forecast information Web service operations

5. Our approach WSMED, a web service based mediator prototype: WSMED mediator SQL query result wrapper DPWSO 1 wrapper DPWSO 2 wrapper DPWSO n SOAP SOAP SOAP WSDL Data Providing Web Service Operations

6. Relational WSMED view View food is based on the web service operation : SearchFoodByDescription select descry from food where gpcode = ’1900’ and keyword = ’Sweet’; SQL Query: WSDL document ≡ ndb keyword descry gpcode 19080 Sweet Candies, Sweet chocolate 1900 ……… . ……… ………… ……… .

11. create view food(ndb, keyword, descry, gpcode) as < wrapper definition >; create view foodclasses(ndb, keyword, gpcode) as select ndb, keyword, gpcode from food; create view fooddescriptions(ndb, descry) as select ndb, descry from food; Multi-level views SQL query accesses the above views: select fd.descry from foodclasses fc, fooddescriptions fd where fc.ndb=fd.ndb and fc.gpcode=’1900’;

13. Comparison of query execution strategies

14. Full semantic enrichment Vs hash join Hash join requires memory to materialize results of the web service calls 0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 0 100 200 300 400 500 600 700 800 900 Number of Food Items Response Time(sec) hash join semantic enrichment

18. Example query select gl.City , gl.TypeId from GetAllStates gs, GetPlacesWithin gp, GetPlaceList gl where gs.state=gp.state and gp.distance=15.0 and gp.placeTypeToFind='City' and gp.place='Atlanta' and gl.placeName=gp.ToPlace + ', ' + gp.ToState and gl.MaxItems=100 and gl.imagePresence='true' Finds information about places located within 15 km from each City named ’ Atlanta ‘ in all US states Invokes 300 web service calls and r eturns a stream of 360 tuples <City, TypeId> GetAllStates GetPlacesWithin GetPlaceList <state> <ToPlace, ToState> <15,’City’,’Atlanta’> <100,’true’>

19. Parallel plans in WSMED Parallel query plan SQL query Calculus Generator Parallel pipeliner Plan function generator Central plan creator Plan splitter Phase 1 Phase 2 central plan

24. AFF_APPLYP (cont.) 2. Executes a monitoring cycle for each invocation of PF for argument tuple a i in non-leaf node 2.1 After the first monitoring cycle A FF_APPLYP adds p new child processes - an add stage to compare performance change 3. When an added node has several levels of children, recursive init stages of A FF_APPLYP s will produce a binary sub–tree q0 q1 q3 q4 q2 q5 Coordinator Level 1 q7 q9 q8 q10 Level 2 q6 q11

25. AFF_APPLYP (cont.) 4. A FF_APPLYP records per monitoring cycle i the average time t i to produce an incoming tuple from the children 4.1 If t i decreases more than a threshold the add stage is rerun 4.2 If t i increases we either add no more children or run a drop stage that drops one child and its children q0 q1 q3 q4 q2 q5 Coordinator Level 1 q12 q10 Level 2 q6 q11

26. Adaptive results p - number of children added after each monitoring cycle Methods with different p value Non parallel plan Best FF_APPLYP Best AFF_APPLYP

30. Experimental results Experiments with adaptive strategies Relative scalability

34. Contributions of papers A - Answered PA – Partially answered 1. Paper1 - Semantic enrichments 2. Paper II - Adaptive parallelization with dependent calls: AFF_APPLYP 3. Paper III - Adaptive parallelization with dependent & independent calls: PAP 4. Paper IV - Web service query service Research questions Paper I Paper II Paper III paper IV 1. How can web service standards be automatically utilized? A A A 2. How can views of web service operations be automatically generated? PA A A 3. How can query optimization be used to provide efficient and scalable search from web services? PA PA A 4. How can the query optimizer speed up queries without any cost estimate? PA PA A 5. How can data sources that are not accessible via web services be transformed into web services? A 6. How can Everything as a Service paradigm be used for querying web services? A