1. XML Programming in PL/SQL (Part 1)“An Introduction to the Oracle XML Database” Marco Gralike

2. XML Programming in PL/SQL

5. Disclaimer The following are “Rules of Numb” Every environment has its own unique criteria and needs… Pay attention to: “Maintainability” “Extendibility” “Choice” “Design” “Testing” “Performance”

6. “XML is not a ‘fast’ thing, there is a ton of parsing involved. Sorry, I never saw the point in huge XML files – they are many times larger than they should be and the amount of work involved in parsing them is incredible”. Tom Kyte - Januari 9, 2009, AskTom

7. If you’re a performance nerd, this is actually cool… Lots of ground to cover regarding “XML”… Solving the customer problem… Back to basics… New solutions via XML Deeper understanding of the data handling issues… So why the “….” XML…?

8. XML Programming in PL/SQL

9. Oracle XML Offering Oracle Apps Oracle Grid Control Oracle Fusion Middleware Oracle Database J2EE Integration Wireless RAC WebService WebCache BI & Reports DataPump OracleXML DB Portal OID LDAP C/C++ XML Utilities (oraxml10.lib) XSQL TransX Java XML Utilities (xml.jar) (XSU) JSR170 Class Generator XMLDiff XMLDiff JAXB XML Pipeline Oracle JDeveloper Java XML Processing (xmlparserv2.jar) C XML Processing (oraxml10.lib) XSLT VM XSD SAX DOM DOM SAX JAXP XPath XQuery XSLT XSD Pull XPath Oracle XML Broadband Network— Binary XML Multi-Platform, Multi-OS Runtime in Java C and C++ NLS (oranls.lib) JDK 1.5 CORE (oracore10.lib) NLS (orai18n*.jar) JDBC (classes12.jar)

10. History 11g XDB 10gR2 10g Performance, Strictness to Standards 9iR2 XDK 9i 8i XMLType, XSQL XQuery Robustness

11. XDK or XMLDB ? Oracle XML Developer's Kit (XDK) Is a versatile set of components that enables you to build and deploy C, C++, and Java software programs that process XML Inside and Outside the database Oracle XMLDB Oracle XML DB is the name for a set of Oracle Database technologies related to high-performance XML storage and retrieval Is officially supported since version 9.2.0.3.0

12. XML DB - BasedonStandards World Wide Web Consortium(W3) XML, XML Schema, XSLT, XPath, XQuery, (11gR1) XLink, XInclude International Standard Organisation(ISO) SQL/XML Internet Engineering Task Force (IETF) FTP, HTTP(s), WebDAV Java Community Process Content Repository API for Java, JSR-170 standard

14. More than one Option

15. Complies to Standards

16. Still License Free

17. Core part of (OXJR)DBMS

19. Milestones – Oracle 9.2/10.1 Oracle Version 9.2.0.3.0 XML DB Build-In instead of a XDK / XMLType CLOB and Object Relational XMLType Storage Oracle 10gR1 Performance, W3C (etc) strict behavior (namespaces!) XML Schema Evolution (copyEvolve)

20. Milestones – Oracle 10.2 Oracle 10gR2 XQuery and XMLTABLE (XPath V2) Query re-write support for eg. xmltype.extract insertChildXML, appendChildXML, insertXMLbefore, deleteXML XMLType support for Advanced Queueing Oracle 10.2.0.3.0 XQuery (also in XMLTABLE) in Database Kernel (build-in via C instead of Java wrappers)

21. Milestones – Oracle 11.1 Oracle 11gR1 XMLType Binary XML Storage Support Unstructured XMLIndex Support DeprecatedCTXXPATH index Repository Events, NDWS Xlink & Xinclude and standard W3C XQuery 1.0 Recommendation In-Place XML Schema Evolution 64K limit on text nodes and attribute values has been lifted

22. Milestones – Oracle 11.2 Oracle 11.1.0.7.0 Oracle Reference or Equi-Partitioning Support Oracle 11.2.0.1.0 Structured XML Index Ability to support huge XML Schema’s Depricated propriety SQL/XML extract, extractvalue, existsnode, … Oracle 11.2.0.2.0 Binary XML default for XMLType (instead of CLOB) Depricated?: ora:view

23. Oracle XML DB SQL*Net Protocol Server Thin, Thick Clients XMLDB Functionality

24. XML Data Handling XML data exchange SQL, C, C++, Java, PL/SQL, .Net, etc Via Protocol Listener supported methods XML data storage CLOB, Object Relational, Binary XML (11gRx) XML data validation XMLSchema, programmatically XML data creation XML Operators, Packages

25. XML DB Mayor Features Storagebasedon the XMLType datatype XMLType Column, XMLTypeTable Retrievalof data via XML/SQL, XPath, XQuery Securitybasedon ACL, Oracle Roles The Protocol Server HTTP(s), FTP, WebDAV, Native Database WebServices (NDWS) The XMLDB Repository XMLSchema Support and Evolution Versioning, CMS Features

26. XML Programming in PL/SQL

27. Document contra Data Driven

30. In Memory: Common XML Parsers Often handle XML tree traversals only via ONEmethod It is not structured, semi-structured or unstructured XML content aware It is not very “smart” / “content aware” regarding XMLhandling based on its XML tree’s and/or XML data content

31. The “Dimensions” in 1 XML doc. 1 3 4 5 2 X Y 6 Z nx rows Elements with maxoccurs=“unbounded”

32. XMLType XML Container (in memory or via storage) In Memory (document) CLOB (document) Object Relational (data) Binary XML (data)

33. Complexities of a database “Relations” “Redundancy” “Nullology” Design, etc… It can contain a database 10 Mb or bigger nowadays More often than less… Enormous complex XSD’s XMLType – Not just a “Container”

34. Checked on XML Well-Formedness One root element Begin & End tags If XML Schema reference XOB methods will be used if an XML Schema is available (faster, smaller/less data) DOM methods will be used if registered XML Schema information is not available XMLType – Not just a “Datatype”

35. XML Operators & Functions DELETEXML EXTRACTVALUE EXISTNODE EXTRACT INSERTCHILDXML INSERTXMLBEFORE SYS_XMLAGG SYS_XMLGEN UPDATEXML APPENDCHILDXML XMLAGG XMLCAST XMLCDATA XMLCOLATTVAL XMLCOMMENT XMLCONCAT XMLDIFF XMLELEMENT XMLEXISTS XMLFOREST XMLPARSE XMLPATCH XMLPI XMLQUERY XMLROOT XMLSEQUENCE XMLSERIALIZE XMLTABLE XMLTRANSFORM etc.,etc.,etc.…

36. XML Operators & Functions…and Packages HTTPURITYPE XDBURITYPE DBURITYPE DBMS_XMLSCHEMA DBMS_XMLDOM DBMS_XMLQUERY DBMS_XMLPARSER DBMS_XMLGEN DBMS_XMLSAVE DBMS_XMLSTORE DBMS_XSLPROCESSOR DBMS_XMLTRANSLATIONS DBMS_METADATA… XMLVIEW XMLTYPE… Handles almost any datatype CLOB, VARCHAR2, ADT,SYS_REFCURSOR, ANYDATA, BLOB, BFILE xmltype.createXML() xmltype.toobject() xmltype.transform()

37. Wow, so what to choose…? Performance C based or Java based solutions Deprecated in the near Future Follow the general XML standard Engine enhancements XPath V2 (XMLTable, XMLQuery) Binary XML support Let Oracle do the Work “Am ”||” i “||” stupid?”,CLOB instead of XMLType

38. Java Based or C Kernel embedded? XQuery (10.2.0.1) 8i related: XMLDOM, XMLPARSER, XMLProcessor DBMS_XMLQUERY DBMS_XMLGEN DBMS_XMLSAVE Pre 11.2.0.0.0 (JVM) ? XQuery (10.2.0.3.0 ) DBMS_XMLDOM DBMS_XMLPROCESSOR XQuery, DBMS_XMLGEN SQL/XML DBMS_XMLSTORE Post 11.2.0.1.0 (no JVM) ?

39. Deprecated in 11.2.0.1.0 EXTRACT EXTRACTVALUE EXISTNODE XMLSEQUENCE ora:instanceof ora:instanceof-only getStringVal(), getClobVal(), getBlobVal() getNamespace() getRootElement() Function-based Indexes (XML) XMLQuery XMLTable, XMLCast, XMLQuery XMLExists XMLTable XQuery instance operator XML Schema attribute xsi:type XMLSerialize XQuery function fn:namespace-uri XQuery function fn:local-name Structured XMLIndex

40. XML Programming in PL/SQL

42. XDB Repository declare res BOOLEAN; begin res := dbms_xdb.createResource ('/public/q1.xqy', <My Xquery>); commit; end; / SELECT xmlquery(xdburitype('/public/q1.xqy').getClob() passing OBJECT_VALUE returning content) FROM MY_XML_TABLE /

43. SQL> SELECT xdata 2 FROM (XMLTABLE( 3 '*' 4 PASSING 5 (XMLTYPE(bfilename('XMLSTORE','data.xml') 6 ,NLS_CHARSET_ID('AL32UTF8'))) 7 COLUMNS xdataXMLTYPE path '/*')); XDATA ------------------------------------ <root><content>Text</content></root> 1 row selected. XML data from your Local Disk

44. SQL> SELECT * 2 FROM XMLTABLE 3 (XMLNAMESPACES('http://purl.org/dc/elements/1.1/' 4 as “MGR") 5 ,'//item' 6PASSING HTTPURITYPE('http://my.blog.com/blog/?feed=rss2').getXML() 7 COLUMNS title varchar2(50) 8 path'/item/title/text()', 9 link varchar2(50) 10 path '/item/link/text()', 11 publication_datevarchar2(50) 12 path '/item/pubDate/text()', 13 creator varchar2(50) 14 path '/item/MGR:creator/text()', 15 description varchar2(250) 16 path '/item/description/text()'); RSS Example (1)

45. TITLE LINK PUBLICATION_DATE CREATOR ------------------------------------------------------------ DESCRIPTION ------------------------------------------------------------ Quering RSS Feeds The XMLDB Way http://feeds.feedburner.com/~r/Bloggralikecom/~3/3 Wed, 25 Jun 2008 16:47:19 +0000 Marco Gralike Actually this IS old stuff (2006), but it got lost IN a… RSS Example (2)

46. SQL> SELECT table_name 2 , to_number( 3 extractvalue(xmltype(dbms_xmlgen.getxml 4 ('select count(*) C 5 from '||table_name)) 6 ,'/ROWSET/ROW/C')) count 7 FROM user_tables; TABLE_NAME COUNT ------------------------------ ------ DEPT 4 EMP 14 BONUS 0 SALGRADE 5 Using XML operators with SQL Source Laurent Schneider:How do i store the counts of all tables …

47. XQuery on PL/SQL variable DECLARE v_x XMLType; NumAcc NUMBER; BEGIN v_x := XMLType(…); /* initialize xmltype variable */ SELECT /*+ NO_XML_QUERY_REWRITE */ XMLCAST(XMLQUERY('declare default element namespace "http://custacc";for $cust in $cadoc/Customer return fn:count($cust/Addresses/Address)' PASSINGv_x AS "cadoc" RETURNING CONTENT) AS NUMBER) INTO NumAcc FROM DUAL; END; Hint allows efficient DOM based evaluation XMLExists() can be used similarly Source OOW 2010 “Managing XML Content with XML DB: Getting the Best Bang for the Buck”

48. Select (XML) data VIEW DBMS_XMLDOM DBMS_XMLPARSER XQUERY

49. XML Programming in PL/SQL

50. When to use XMLType Query against XML data Take advantage of structure when you have an XML schema Provide efficient XPath access Shield applications from storage models Prepare for future optimizations Provide structured storage with DOM fidelity

51. Create XML Relational Content XMLType XMLView XQuery DBMS_XML{Packages} Object Orientated Content XML Content Document Content BFilename, BLOB, CLOB, EXIF…

52. Canonical XMLType Result Set XMLType - Very powerful (…and simple) SQL> select xmltype(cursor(select * fromscott.dept 2whererownum=1)) 3 as "Result" 4fromdual; Result --------------------------------------------------------- <?xmlversion="1.0"?> <ROWSET> <ROW> <DEPTNO>10</DEPTNO> <DNAME>ACCOUNTING</DNAME> <LOC>NEW YORK</LOC> </ROW> </ROWSET>

53. SQL/XML XMLELEMENT, XMLATTRIBUTES, XMLFOREST, XMLSEQUENCE, XMLCONCAT, XMLAGG XMLTABLE, XMLQUERY XMLEXIST, XMLCAST SYS_XMLGEN, SYS_XMLAGG DBMS_XML{PACKAGES}

54. Master – Detail (1) SELECTdepartment_id as "ID", XMLElement("Department", XMLAttributes(d.department_name "name"), (SELECTXMLAgg( XMLElement("emp", XMLAttributes(e.last_name name), (SELECTXMLAgg( XMLElement("jobs", XMLAttributes(j.job_id "job"))) FROMjob_history j WHERE j.employee_id=e.employee_id))) FROM employees e WHEREe.department_id=d.department_id)) AS “MD" FROM departments d WHERE department_id < 40;

55. Master – Detail (2) ID MD --------- --------------------------------------- 90 <Department name="Executive"> <emp NAME="King"/> <emp NAME="Kochhar"> <jobs job="AC_ACCOUNT"/> <jobs job="AC_MGR"/> </emp> <emp NAME="De Haan"> <jobs job="IT_PROG"/> </emp> </Department> …rows selected.

56. Native XML Generation Generate XML documents by using : SYS_XMLGEN, to accept a single argument and generate an XML document for each row from a SQL statement SYS_XMLAGG, to process groups of rows, aggregating several XML fragments into a single XML document In PL/SQL, use the DBMS_XMLGEN package to execute any SQL query and map the results into an XML document as a CLOB or XMLType.

57. SYS_XMLGEN() Function Converts its argument into an XML document Accepts two arguments: A scalar value, object type, or XMLType instance An optional XMLFormat() object Returns an XMLType instance Creates and queries XML instances within SQL SELECT SYS_XMLGEN( XMLForest (last_name, salary), XMLFormat('EMPLOYEE')) FROM employees WHERE department_id = 30;

58. SYS_XMLAGG() Function The SYS_XMLAGG() function: Groups all the input documents and produces a single XML document Is used to aggregate (concatenate) fragments SELECT SYS_XMLAGG( SYS_XMLGEN(last_name), XMLFormat('Employees')) result FROM employees WHERE department_id < 30 GROUP BY department_id;

59. DBMS_XMLGEN Creates an XML document from SQL query, XMLType Gets the document as a CLOB Provides a fetch interface Is useful for pagination in Web applications

60. DBMS_XMLGEN (1) CREATE TABLE temp_clob_tab(result CLOB); DECLARE qryCtx DBMS_XMLGEN.ctxHandle; result CLOB; BEGIN qryCtx := DBMS_XMLGEN.newContext('SELECT * FROM hr.employees'); -- Set the row header to be EMPLOYEE DBMS_XMLGEN.setRowTag(qryCtx, 'EMPLOYEE'); -- Get the result result := DBMS_XMLGEN.getXML(qryCtx); INSERT INTO temp_clob_tab VALUES(result); --Close context DBMS_XMLGEN.closeContext(qryCtx); END; /

61. DBMS_XMLGEN (2) RESULT ------------------------------------------------ <?xml version="1.0"?> <ROWSET> <EMPLOYEE> <EMPLOYEE_ID>100</EMPLOYEE_ID> <FIRST_NAME>Steven</FIRST_NAME> <LAST_NAME>King</LAST_NAME> <EMAIL>SKING</EMAIL> <PHONE_NUMBER>515.123.4567</PHONE_NUMBER> <HIRE_DATE>17-JUN-87</HIRE_DATE> <JOB_ID>AD_PRES</JOB_ID> <SALARY>24000</SALARY> <DEPARTMENT_ID>90</DEPARTMENT_ID> </EMPLOYEE> … </ROWSET>

62. Create XML data DBMS_XMLDOM SYS _XMLGEN XMLVIEW

63. XML Programming in PL/SQL

64. National LanguageSettings CHARACTER SET: AL32UTF8 NATIONAL CHARACTER SET: AL16UTF16 NLS_LENGTH_SEMANTICS ='BYTE'; Before $ORACLE_HOMEDBMSDMINatqm.sql Avoid “Strange” NLS related errors: Registering an XML Schema via WebDAV, Drag & Drop Registering an XML Schema via BFILENAME

65. Database character set is AL32UTF Client character set is WIN1252. SQL> drop table test; SQL> create table test of xmltype; SQL> insert into test values (xmltype( '<?xml version="1.0" encoding="UTF-8"?><FOO/>')); SQL> select object_value from test; OBJECT_VALUE --------------------------------------------------- <?xml version="1.0" encoding="WINDOWS-1252"?><FOO/> NLS FAQ (1)

66. SQL> select x.object_value.getClobVal() from test x; X.OBJECT_VALUE.GETCLOBVAL() -------------------------------------------- <?xml version="1.0" encoding="UTF-8"?><FOO/> SQL> exit C:EMP>set NLS_LANG=AMERICAN_AMERICA.AL32UTF8 C:EMP>sqlplusscott/tiger SQL> select * from test; SYS_NC_ROWINFO$ -------------------------------------------- <?xml version="1.0" encoding="UTF-8"?><FOO/> NLS FAQ (2)

67. ORA-24247 ORA-24247 Network access denied by access control list, ACL Packages involved (11gR1) UTL_TCP, UTL_SMTP, UTL_MAIL, UTL_HTTP, UTL_INADDR Solution: DBMS_NETWORK_ACL_ADMIN Check, Add, Remove Privileges

68. Troubleshooting XMLDB Use What You Know: SET AUTOTRACE ON, DBMS_XPLAN.DISPLAY, TKPROF, SQL_TRACE, EVENTS like 10046 etc. Schema Registration: ALTER session SET events = '31098 trace name context forever' Protocol Server: ALTER system SET event = '31098 trace name context forever, level 2' scope=spfile Trace files  UDUMP Directory

69. Small tip regarding “alias” usage Using an alias called: “X”, “Y” (and not Z…) SQL> select * from v$versionwhererownum=1; BANNER --------------------------------------------------------- Oracle Database 10g EnterpriseEdition Release 10.2.0.4.0 SQL> describeX FUNCTION X RETURNS NUMBER Argument Name Type In/Out Default? ------------------------------ ----------------------- ------ -------- P ST_POINT IN SQL> describeY FUNCTION Y RETURNS NUMBER Argument Name Type In/Out Default? ------------------------------ ----------------------- ------ -------- P ST_POINT IN

70. Reward True understanding Optimal performance Out performing XML (Java) More options from “within” XMLDB: a “No-Cost” option Opening up propriety environments

72. References (1) Oracle XML DB http://download.oracle.com/docs/cd/E11882_01/appdev.112/e16659/toc.htm XML DB OTN / FAQ Thread http://forums.oracle.com/forums/forum.jspa?forumID=34 http://forums.oracle.com/forums/thread.jspa?threadID=410714

73. References (2) Oracle Whitepapers Oracle XML DB : Choosing the Best XMLType Storage Option for Your Use Case (PDF) Oracle XML DB : Best Practices to Get Optimal Performance out of XML Queries (PDF) Blog http://technology.amis.nl/blog http://blog.gralike.com (Dedicated XMLDB blog)