An Introduction to Graph: Database, Analytics, and Cloud Services

1. Analytics and Data Summit 2018 An Introduction to Graph: Database, Analytics, and Cloud Services Jean Ihm Hans Viehmann Jan Michels Product Manager US Product Manager EMEA Principal Member of Technical Staff Oracle Oracle Oracle @JeanIhm @SpatialHannes

2. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 2

5. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 5 https://www.macalester.edu/~abeverid/thrones.html A social network generated from A Storm of Swords. The color of a vertex indicates its community. The size of a vertex corresponds to its PageRank value, and the size of its label corresponds to its betweenness centrality. An edge’s thickness represents its weight.

6. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Oracle’s Spatial and Graph Strategy Enabling Spatial and Graph use cases on every platform 6 Oracle Big Data Spatial and Graph Oracle Database Spatial and Graph Spatial and Graph in Oracle Cloud Database Cloud Service Exadata Cloud Service

7. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Two Graph Data Models RDF Data Model • Data federation • Knowledge representation • Semantic Web Social Network Analysis  Financial  Retail, Marketing  Social Media  Smart Manufacturing Linked Data Semantic Web Property Graph Model • Path Analytics • Social Network Analysis • Entity analytics  Life Sciences  Health Care  Publishing  Finance Use Case Graph Model Industry Domain RDF Use Case Sessions: Linked open data initiatives from Ireland/Japan, tax fraud detection in Florence, Italy – Wednesday 1:00-2:45

8. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Graph Database Features: • Scalability, Performance, Security • Graph analytics • Graph Visualization • Graph Query Language • Standard interfaces • Integration with Machine Learning tools 8 Courtesy Tom Sawyer Perspectives Courtesy Linkurious

9. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Oracle Big Data Spatial and Graph • Available for Big Data platform – Hadoop, HBase, Oracle NoSQL • Supported both on BDA and commodity hardware – CDH and Hortonworks • Database connectivity through Big Data Connectors or Big Data SQL • Included in Big Data Cloud Service Oracle Spatial and Graph (Database) • Available with Oracle 12.2/18.1 • Using tables for graph persistence • Graph views on relational data • In-database graph analytics – Sparsification, shortest path, page rank, triangle counting, WCC, sub graphs • SQL queries possible • Included in Database Cloud Service 9 Graph Product Options

12. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Some Use Case Scenarios • Finance – Customer 360, Fraud detection • Public Sector – Tax Evasion, Crime network analysis • Retail – Recommendation, sentiment analysis • Manufacturing – Analyzing complex bill of materials (BoM)

13. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Financial Services • Model customer relationship to products, services, people, places. • Analyze money customer’s flow between non-bank to bank accounts • Combine internal CRM data with enterprise and social media content • Identify high-value customers across business divisions • Enhance new product/service opportunities • Provide Real-time recommendations 13 Applying Graph Analysis To Improve Customer Service Use Case Session: Follow the Money: A Graph Model for Monitoring Bank Transactions - Wednesday 10:05am

15. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Tax Fraud Analysis Chinese Province Tax Office Challenge: – Modeling relationships between individuals and corporations – Ingest documents, social media, web content, and publicly available open data – Create a ‘picture’ of the taxpayer network • Taxpayer relationship with other taxpayers • If a company structure, identify associated directors and shareholders in that company • Relationship between taxpayer’s and their associates’ financial affairs • Identify relevant intermediaries acting on behalf of taxpayer – Explore tax evasion and fraud, trigger a formal case investigation 15

16. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Public Security: Analyzing Criminal Networks Chinese Police Department Business Requirement – Model relationships between known and suspected criminals – Ingest documents, social media, web content, chat rooms, flight records, hotel stay registries, and publically available open datasets. How graph analysis solves the problem • Search for known individuals in web of content • Analyze relationship with other criminals, travel history, addresses, employers • Relationship between suspects and their financial affairs Courtesy Tom Sawyer Perspectives Use Case Session: Construct a Public Security Data Analysis Engine Using Oracle Big Data Solution (Justin Zou) Wednesday 10:35am

17. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Analyzing Blockchain Ledger Transactions • Distributed Ledgers being adopted in Finance, Public Sector • Load and manage massive transactions from a distributed digital ledger • Efficiently traverse a blockchain transaction graph • Query and visualize – search for patterns of activity 17 Land Management, Banking, Public Services Tech Session: Analyzing Blockchain and Bitcoin Transaction Data as Graph (Z. Wu / X. Lopez) Thursday 8:45am

18. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | CyberNetwork Modeling & Analysis • Model existing cloud infrastructure entities as a connected graph – servers, switches, storage, firewalls… • Analysis of network behavior – System impact of change to network topology – Reachability analysis to understand which routines, libraries, servers, routers, etc., are affected by a modification – Nework intrusion detection (next slide) Monitoring Cloud Infrastructure Services

19. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 19 Automotive Manufacturing Support high variance, short innovation cycles of complex autos Voice of Customers Model Based Design Configuration & BOM Simulation 3D CAD & CAE Factory Resources Financial Requirements • Unified graph representation of BoM, Configuration, CAE, Simulation... • Generate “graph view” of relational data, or model instance data as graph • Apply graph query and search across BoM and configuration models • Apply graph analytics • Scale to trillions of nodes and edges Graph View of Enterprise Data Use Case Session: Graph Modeling and Analysis for Complex Automotive Data Management (Mazda) Wednesday 11:10 am

21. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Oracle Graph Analytics Architecture Scalable and Persistent Storage Graph Data Access Layer API Graph Analytics In-memory Analytic Engine Blueprints & SolrCloud / Lucene Property Graph Support on Apache HBase, Oracle NoSQL or Oracle 12.2 RESTWebService Python,Perl,PHP,Ruby, Javascript,… Java APIs Java APIs/JDBC/SQL/PLSQL 21 CytoscapePlug-in RIntegration(OAAgraph) Sparkintegration SQL*Plus, …

22. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Interacting with the Graph • Access through APIs – Implementation of Apache Tinkerpop Blueprints APIs – Based on Java, REST plus SolR Cloud/Lucene support for text search • Scripting – Groovy, Python, Javascript, ... – Apache Zeppelin integration, Javascript (Node.js) language binding • Graphical UIs – Cytoscape, plug-in available – Commercial Tools such as TomSawyer Perspectives On-premise product geared towards data scientists and developers 22

23. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Graph Analysis Algorithms can be very hard to code ... • Example: Find the size of the 2-hop network of vertices (Gremlin+Python) • Single API call instead – Analysis in memory, in parallel Oracle Big Data Spatial and Graph comes with 40+ pre-built algorithms sum([v.query() .direction(blueprints.Direction.OUT).count() for v in OPGIterator(v0.query() .direction(blueprints.Direction.OUT) .vertices().iterator())])

24. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Example: Betweenness Centrality in Big Data Graph Code 24 D A C E B F I J H K G analyst.vertexBetweennessCentrality(pg) .getTopKValues(15)

25. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Structure Evaluation – Conductance – countTriangles – inDegreeDistribution – outDegreeDistribution – partitionConductance – partitionModularity – sparsify – K-Core computes Community Detection – communitiesLabelPropagation Ranking – closenessCentralityUnitLength – degreeCentrality – eigenvectorCentrality – Hyperlink-Induced Topic Search (HITS) – inDegreeCentrality – nodeBetweennessCentrality – outDegreeCentrality – Pagerank, weighted Pagerank – approximatePagerank – personalizedPagerank – randomWalkWithRestart 25 Graph Analysis Algorithms (1)

26. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Pathfinding – fattestPath – shortestPathBellmanFord – shortestPathBellmanFordReverse – shortestPathDijkstra – shortestPathDijkstraBidirectional – shortestPathFilteredDijkstra – shortestPathFilteredDijkstraBidirectional – shortestPathHopDist – shortestPathHopDistReverse Recommendation – salsa – personalizedSalsa – whomToFollow Classic - Connected Components – sccKosaraju – sccTarjan – wcc 26 Graph Analysis Algorithms (2)

27. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | • Multi-purpose notebook for data analysis and visualization – Default for Oracle Cloud Services • Enables to embed interactive execution inside Browsers • Renders execution results with plots and tables within Browsers • PGX provides a hook (interpreter) for Zeppelin integration 27 Apache Zeppelin integration

28. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Combining Graph Analytics and Machine Learning Graph Analytics • Compute graph metric(s) • Explore graph or compute new metrics using ML result Machine Learning • Build predictive model using graph metric • Build model(s) and score or classify data 28 Add to structured data Add to graph

29. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | OAAgraph integration with R • OAAgraph integrates in-memory engine into ORE and ORAAH • Adds powerful graph analytics and querying capabilities to existing analytical portfolio of ORE and ORAAH • Built in algorithms of PGX available as R functions • PGQL pattern matching • Concept of “cursor” allows browsing of in-memory analytical results using R data structures (R data frame), allows further client-side processing in R • Exporting data back to Database / Spark allows persistence of results and further processing using existing ORE and ORAAH analytical functions Dedicated session by Mark Hornick and Sungpack Hong in Big Data track

30. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | SPARK SPARK SPARK PGX PGX Graph Analytics on SPARK • Use SPARK for conventional tabular data processing (RDD, Dataframe, -set) • Define graph view of the data – View it as node table and edge table • Load into PGX • Execute graph algorithms in PGX – Orders of magnitude faster than GraphX – More scaleable • Push analysis results back into SPARK as additional tables • Continue SPARK analysis SPARK data structure and communication mechanism not optimized for graph analysis workloads

31. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 31 PGX Performance (Query) vs. Neo4j Hardware: 88 x Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz - 256 RAM Warmup: ignore the first two runs, measure the third run Index usage: no special indexes (use defaults) 1 10 100 1000 10000 100000 1000000 10000000 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 Time(ms) Query Code Graph Dataset: 5M nodes and 64M edges Neo4j 3.0.7 PGX 2.3.0 76x 142x 71x 8x 4x 22x 57x 30x 15x 2x 12x 613x 307x 34X > 800,000x 4x 20x Recursive path queries Numbers shown are “hot” numbers: Neo4j is no longer accessing disk.

32. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Distributed Graph Analysis Engine • Oracle Big Data Spatial and Graph uses very compact graph representation – Can fit graph with ~23bn edges into one BDA node • Distributed implementation scales beyond this – Processing even larger graphs with several machines in a cluster (scale-out) – Interconnected through fast network (Ethernet or, ideally, Infiniband) • Integrated with YARN for resource management – Same client interface, but not all APIs implemented yet • Again, much faster than other implementations – Comprehensive performance comparison with GraphX, GraphLab Handling extremely large graphs

33. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | • SQL-like syntax but with graph pattern description and property access – Interactive (real-time) analysis – Supporting aggregates, comparison, such as max, min, order by, group by • Finding a given pattern in graph – Fraud detection – Anomaly detection – Subgraph extraction – ... • Proposed for standardization by Oracle – Specification available on-line – Open-sourced front-end (i.e. parser) Pattern matching using PGQL https://github.com/oracle/pgql-lang 33

34. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | PGQL Example query – PGQL 1.1 • Find all instances of a given pattern/template in data graph • Fast, scaleable query mechanism SELECT v3.name, v3.age FROM socialNetworkGraph MATCH (v1:Person) –[:friendOf]-> (v2:Person) –[:knows]-> (v3:Person) WHERE v1.name = ‘Amber’ query Query: Find all people who are known to friends of ‘Amber’. 34 socialNetwork Graph 100 :Person name = ‘Amber’ age = 25 200:Person name = ‘Paul’ age = 30 300 :Person name = ‘Heather’ age = 27 777 :Company name = ‘Oracle’ location = ‘Redwood City’ :worksAt{1831} startDate = ’09/01/2015’ :friendOf{1173} :knows{2200} :friendOf {2513} since = ’08/01/2014’

35. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Graph Query Language Standardization – Status Quo • Why Property Graphs with SQL? • Who is involved in the standardization of SQL extensions for Property Graphs? • What might the SQL extensions for Property Graphs look like? The SQL Standard and Property Graphs

36. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Why Property Graphs with SQL? • Users are using both SQL data and Property Graph data • High interest in PGs from: – Users – Vendors • Building consensus among the players on: – Syntax – Semantics • Application development is easier, better, quicker, faster if only one interface

37. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Who is involved? • ISO: – JTC 1/SC32/WG3 – USA, Germany, Japan, UK, Canada, China • ANSI: – INCITS / DM32 / DM32.2 / DM32.2 Ad Hoc Group on SQL Extensions for Property Graphs – Oracle, Neo4j, Microsoft, IBM, SAP/Sybase, Teradata, JCC Consulting, Actian Aka the ISO SQL committee Aka the US SQL committee

38. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | SQL extensions for Property Graphs (PGs) • Goal: define extensions to query property graphs – Agree on one (or possibly more) representation of PGs in SQL • Most obvious, in tables • Maybe later, some “native” storage format – Agree on the way to query PGs in SQL • Query PGs “natively” (use the power of pattern matching) – Maybe later embed (full) SQL in PG queries • Represent result as a table (unleash the power of SQL on the result) • Maybe later DML operations on a property graph directly • Targeted for SQL:2020 – No concrete proposal/agreement yet

39. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Property Graph Definition (DDL) – Example • Example: CREATE PROPERTY GRAPH myGraph VERTEX TABLES (Person, Message) EDGE TABLES ( Created SOURCE Person DESTINATION Message, Commented SOURCE Person DESTINATION Message ) • Existing tables (or views): Person, Message, Created, Commented • Implementation can infer keys from (primary/foreign keys) of underlying tables • All columns of each table are exposed as properties of the corresponding vertex/edge (tables) Create a PG w/two vertex tables and two edge tables.

40. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | DDL – Example (cont.) • Example for optional clauses CREATE PROPERTY GRAPH myGraph VERTEX TABLES ( People KEY ( id ) LABEL Person PROPERTIES ( emailAddress AS email ), Messages KEY ( id ) LABEL Message PROPERTIES ( created as creationDate, content ) ) EDGE TABLES ( CreatedMessage KEY ( id ) SOURCE KEY ( creator ) REFERENCES People DESTINATION KEY ( message ) REFERENES Messages LABEL Created NO PROPERTIES, CommentedOnMessage KEY ( id ) SOURCE KEY ( commenter ) REFERENCES People DESTINATION KEY ( message ) REFERENCES Messages LABEL Commented NO PROPERTIES ) Same PG as before – but fine-grained control over labels, properties, etc.

41. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Querying PGs - Example SELECT GT.creationDate, GT.content FROM myGraph GRAPH_TABLE ( MATCH (Creator IS Person WHERE Creator.email = :email1) -[ IS Created ]-> (M IS Message) <-[ IS Commented ]- (Commenter IS Person WHERE Commenter.email = :email2) WHERE ALL_DIFFERENT (Creator, Commenter) ONE ROW PER MATCH COLUMNS ( M.creationDate, M.content ) ) AS GT Get the creationDate and content of the messages created by one person ("email1") and commented on by another person ("email2").

42. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Using PGQL with the Oracle Database • New feature in Oracle 18c • Provides easy and natural way to express graph queries – SQL translation is performed automatically behind the scenes – Users don’t need to worry about writing complex SQL • Querying data on disk – Store graphs which do not fit into memory – Dynamic data can be queried without the need to constantly push updates to in- memory snapshots – Joint analysis with other data in the database 42 PGQL-to-SQL conversion in Oracle Spatial and Graph

44. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Graph Visualization – Commercial Tools Dedicated session on TomSawyer Perspectives integration 48 TomSawyer Presentation: Analyze the Global Knowledge Graph – Wednesday 9:00-9:50

45. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | • A new PGX client – Based on JavaScript (Node.JS) – Support both PGX shared memory and distributed runtime • Design highlights – Leveraging latest ES6 language features – ES6 classes and getters – Arrow Functions – Chaining of Promises – Clean error handling Node.js Client PGX shared memoryNode.js PGX Client https RESTInterface PGX distributed const p = pgx.connect("http://myhost:7007", options) p.then(session => { return session.readGraphWithProperties(json) }).then(graph => { let analyst = graph.session.analyst return analyst.pagerank(graph, 0.001, 0.85, 100) }).then(rank => { rank.topK(5).foreach(elem => console.log(elem)) let q = "SELECT n, n.pagerank WHERE (n)" return rank.graph.queryPgql(q) }).catch(function(err) { console.log("error: " + err); }) Code Example:

46. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Linkurious Ogma • Server-based (Node.JS) • Light-weight JavaScript visualizer • Powerful rendering • Oracle integration 50 https://www.slideshare.net/Linkurious/how-to-visualize-oracle-big-data-spatial-and-graph-with-ogma https://linkurio.us/visualize-oracle-graph-data-ogma-library/ https://linkurio.us

48. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Autonomous Graph Cloud Service • Simple: Fully-managed, pre-configured, automatically tuned • Fully Managed: “One-click” deployment, no installation, zero configuration • Comprehensive: Data management, analytics, domain workflows • Developer Ready: Low code /zero code, APIs, and query language • Data Scientist Friendly: Graph Modeler, Studio, and Notebooks Session: Sneak Preview: Graph Cloud Services and Spatial Console for Database Cloud (Jim Steiner, Jayant Sharma) Tuesday 11:15am

49. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Summary Graph capabilities in Oracle Big Data Spatial and Graph • Graph databases are powerful tools, complementing relational databases – Especially strong for analysis of graph topology and multi-hop relationships • Graph analytics offer new insight – Especially relationships, dependencies and behavioural patterns • Oracle Property Graph technology offers – Comprehensive analytics through various APIs, integration with relational database – Scaleable, parallel in-memory processing – Secure and scaleable graph storage using Oracle NoSQL, HBase or Oracle Database • Available both on-premise or in the Cloud 53

50. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Analytics and Data Summit – Graph Sessions Tuesday • An Introduction to Graph: Database, Analytics, and Cloud Services (Ihm, Viehmann, Michels) 10:10am • Sneak Preview: Graph Cloud Services and Spatial Console for Database Cloud (Jim Steiner, Jayant Sharma) 11:15am Wednesday • Analyze the Global Knowledge Graph with Visualization, Cloud, & Spatial Tech (Tom Sawyer Software) 9:00am • Follow the Money: A Graph Model for Monitoring Bank Transactions (Federico Calabria) 10:05am • Graph Modeling and Analysis for Complex Automotive Data Management (Mazda) 11:10am • Construct a Public Security Data Analysis Engine Using Oracle Big Data Solution (Justin Zou) 11:35am 54 *All sessions in room 103 unless otherwise noted

51. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Analytics and Data Summit – Graph Sessions Wednesday (continued) • A Graph-Enabled Data Discovery Platform Supporting GDPR Compliance (W. Wcislo) 2:45pm • Anomaly Detection in Medicare Provider Data Set using OAAgraph (M. Hornick, S. Hong) 4:30pm Thursday • Analyzing Blockchain and Bitcoin Transaction Data as Graph (Zhe Wu, Xavier Lopez) 8:45am • Ingesting streaming data into Graph database (Trivadis) 9:50am • Fake News, Trolls, Bots and Money Laundering – Find the truth with Graphs (Jim Steiner, Sungpack Hong) 10:55am • Measuring Published Citation using Graph Modeling and Analysis (The Institute of Statistical Mathematics, Tokyo, Japan) 1:55pm 55 *All sessions in room 103 unless otherwise noted

52. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Analytics and Data Summit – Graph Sessions • Hands-on Labs *Held in 500 building room 104 – Property Graph 101 on Oracle Database 12.2 for the completely clueless (Vlamis Software) Tuesday, 3:35pm – Using Property Graph and Graph Analytics on NoSQL DB to analyze data on Meetup.com (Karin Patenge) Wednesday, 2:10pm • RDF Graph sessions on Wednesday – Serving Ireland's Authoritative Geospatial Data as Linked Data (Trinity College Dublin) 1:00pm – Publication of Statistical Linked Open Data in Japan with Oracle RDF graph (National Statistics Center of Japan) 1:25pm – Discovering Tax Fraud in Florence, Italy with a Semantic Graph Database (Evodevo) 2:20pm 56 *All sessions in room 103 unless otherwise noted

53. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Engage with the Spatial and Graph SIG Promotes interaction and communication to drive the market for spatial technology and data Members connect and exchange knowledge via online communities and at conferences and events • Talk with us at the Summit! • • Join us online • tinyurl.com/oraclespatialcommunity • Search for “Oracle Spatial and Graph Community” • Contact us: oraclespatialsig@gmail.com @oraspatialsig Morning Arrivals Tues & Wed 7:45-8:30 a.m. Registration Area Birds of a Feather Lunch Wednesday 12-1pm Auditorium Receptions Tues & Wed evenings Spatial Table, lobby

An Introduction to Graph: Database, Analytics, and Cloud Services

Esté a ler uma amostra.

Confira estes a seguir

An Introduction to Graph: Database, Analytics, and Cloud Services

Mais Conteúdo rRelacionado

Diapositivos para si (20)

Semelhante a An Introduction to Graph: Database, Analytics, and Cloud Services (20)

Mais de Jean Ihm (9)

Mais recentes (20)