SlideShare a Scribd company logo
1 of 47
Welcome                                                       NoSQL and SOA
                                                                      Guido Schmutz
                                                                      10.10.2012




BASEL   BERN   LAUSANNE     ZÜRICH      DÜSSELDORF   FRANKFURT A.M.   FREIBURG I.BR.   HAMBURG   MÜNCHEN   STUTTGART   WIEN




                      2012 © Trivadis
 1
                      NoSQL and SOA
                      10.10.2012
Guido Schmutz

•   Working for Trivadis for more than 15 years
•   Oracle ACE Director for Fusion Middleware and SOA
•   Co-Author of different books
•   Consultant, Trainer Software Architect for Java, Oracle, SOA
    and EDA
•   Member of Trivadis Architecture Board
•   Technology Manager @ Trivadis


•   More than 20 years of software development
    experience


•   Contact: guido.schmutz@trivadis.com
•   Blog: http://guidoschmutz.wordpress.com
•   Twitter: gschmutz

                2012 © Trivadis
2
                NoSQL and SOA
                10.10.2012
Agenda


1. Why NoSQL and what is it?
2. NoSQL Database Types
3. Polyglot Persistence
4. NoSQL and Service-Oriented Architecture
5. Summary




              2012 © Trivadis
3
              NoSQL and SOA
              10.10.2012
History of Database


1960s            File-based, Network (CODASYL) and Hierarchical
Databases
1970s            Relational Database
1980             SQL became the standard query language
Early 1990       Object-Databases
Late 1990        XML Databases
2004             NoSQL Databases




             2012 © Trivadis
4
             NoSQL and SOA
             10.10.2012
What„s wrong with Relational Databases ? They are great ….


•   SQL provides a rich, declarative query language
•   Database enforce referential integrity
•   ACID semantics
•   Well understood by developers, database administrators
•   Well supported by different languages, frameworks and tools
    • Hibernate, JPA, JDBC, iBATIS, Entity Framework
•   Well understood and accepted by operations people (DBAs)
    •   Configuration
    •   Monitoring
    •   Backup and Recovery
    •   Tuning
    •   Design

                2012 © Trivadis
5
                NoSQL and SOA
                10.10.2012
Relational Databases are great ... But!
                                                                                             ORDER
                                                Order
                                                 ID: 1001

Problem: Complex Object graphs
                                                 Order Date: 15.9.2012

                                                  Customer

     Object/Relational impedance mismatch
                                                                                             CUSTOMER
                                                   First Name: Peter
                                                   Last Name: Sample


     Complicated to map rich domain model
                                                    Billing Address
                                                        Street: Somestreet 10

      to relational schema
                                                        City: Somewhere
                                                        Postal Code: 55901                   ADDRESS



     Performance issues                          Line Items


         Many rows in many tables                 Name

                                                   Ipod Touch
                                                                      Quantity

                                                                                1
                                                                                     Price

                                                                                    220.95
                                                                                             ORDER_LINES



         Many joins                               Monster Beat                 2   190.00


         Eager vs. lazy loading                   Apple Mouse                  1    69.90




Problem: Schema evolution
     Adding attributes to an object => have to add columns to table
     Expensive, if lots of data in that table
      - Holding locks on the tables for long time
      - Application downtime …

                 2012 © Trivadis
6
                 NoSQL and SOA
                 10.10.2012
ORDER
                                          Order



Relational Databases are great ... But!
                                           ID: 1001
                                           Order Date: 15.9.2012

                                            Customer
                                                                                       CUSTOMER
                                             First Name: Peter
                                             Last Name: Sample
                                              Billing Address
                                                  Street: Somestreet 10
                                                  City: Somewhere
                                                  Postal Code: 55901                   ADDRESS



                                            Line Items
                                             Name               Quantity       Price   ORDER_LINES

                                             Ipod Touch                   1   220.95

                                             Monster Beat                 2   190.00

                                             Apple Mouse                  1    69.90




                                                  Consumer


                                                              REST/SOAP



                                                  Service


                                                  Repository/DAO


                                                  O/R Mapping



                                                                    SQL




                                                  RDBMS




            2012 © Trivadis
7
            NoSQL and SOA
            10.10.2012
Relational Databases are great ... But!


Problem: Semi-structured data
     Relational schema doesn„t easily handle semi-structured data
     Common solutions
      - Name/Value table
        - Poor performance
        - Lack of constraint
      - Serialize as Blob
        - Fewer joins, but no query capabilities
Problem: Scaling
     Scaling writes difficult/expensive/impossible => BigData
     Vertical scaling is limited and is expensive
     Horizontal scaling is limited and is expensive



                 2012 © Trivadis
8
                 NoSQL and SOA
                 10.10.2012
Solution: NoSQL ?


No standard definition of what NoSQL means
•   Not Only SQL
Term began in a workshop organized in 2009
but some common characteristics of NoSQL databases
•   They don„t use the relational data model and thus don„t use SQL
•   Tend to be designed to run on cluster
                                                                    RDBMS                                NoSQL
•   Tend to be Open Source              Presentation
                                            Tier
                                                                   User Interface                       User Interface




•   Schema-Less - Don„t have a fixed                                                                                           Key Value Stores


    schema, allowing to store any




                                                                                                                                                  Services
                                                                                                  Caching             Search
                                         Middle Tier   Object-Relational    Relational-Object                                      Lucene
                                                                                                Transactions          Batch


    data in any record
                                                                                                                                 MapReduce




•   Different APIs
                                                             Search             Blobs


                                       Database Tier      Transactions          Batch                          Data


                                                            Caching            Triggers




                 2012 © Trivadis
9
                 NoSQL and SOA
                 10.10.2012
Central vs. Application Databases

Central Database                                           Application Database
•    Using SQL as the integration mechanism                •   Only accessed by a single application
     between applications
                                                           •   Only the application using the database
•    applications store data in common DB                      needs to know about the structure
•    Improves communication, all applications              •   Easier to maintain and evolve the schema
     operate on consistent set of data
                                                           •   More freedom to choose the database
•    Structure ends up to be more complex
                                                           •   Applicable to SOA (i.e. Data Service/Entity
•    Changes need to be coordinated with all                   Service) with good Service Autonomy
     other applications using the database
                                                           •   Ready for the cloud
•    Side-effects (i.e. adding database index)

     Application 1   Application 2         Application 3        Application 1   Application 2   Application 3




                         DB                                         DB              DB               DB


                         2012 © Trivadis
10
                         NoSQL and SOA
                         10.10.2012
Relational vs. Aggregate Data Models


 The relational model takes the         Aggregate is a term that comes
  information and divides it into         from Domain-Driven Design
  tuples (rows)                           (Evans)
 A tuple is a limited data structure    An aggregate is a collection of
      no nesting of tuples               related objects, that should be
      no list of values                  treated as a unit
                                           Unit for data manipulation and
                                            management of consistency




                  2012 © Trivadis
11
                  NoSQL and SOA
                  10.10.2012
Relational vs. Aggregate Data Model


Relational Instance                                             Aggregate Instance
CUSTOMER                                  PRODUCT
 ID    NAME                                ID       NAME        {
  1    Guido                              1000    IPod Touch    „id“:1,
                                          1020   Monster Beat   „name“:“Guido“,
 BILLING_ADDRESS                                                „billingAddress“:[{„street“:“Chaumontweg“,“city“:“Spiegel“,“postCode“:“3095“}]
                                                                }
 ID      CUSTOMER_ID        ADDRESS_ID
  1           1                 55                              {
                                                                „id“:90,
      ADDRESS                                                   „customerId“:1,
 ID         STREET               CITY          POST_CODE        „orderItems“:[
                                                                  {
 55       Chaumontweg           Spiegel           3095
                                                                  „productId“:1000,“price“: 250.55, „produtName“: „iPod Touch“
                                                                  },
 ORDER                                                            {
 ID    CUSTOMER_ID         SHIPPING_ADDRESS_ID                    „productId“:1020,“price“: 199.55, „produtName“: „Monster Beat“
 90         1                       55                            }],
                                                                „sippingAddress“:[{„street“:“Chaumontweg“,“city“:“Spiegel“,“postCode“:“3095“}]
                                                                }
ORDER_ITEM
 ID       ORDER_ID           PRODUCT_ID         PRICE
  1          90                 1000            250.55
  1          90                 1020            199.55




                        2012 © Trivadis
12
                        NoSQL and SOA
                        10.10.2012
Agenda


1. Why NoSQL and what is it?
2. NoSQL Database Types
3. Polyglot Persistence
4. NoSQL and Service-Oriented Architecture
5. Summary




              2012 © Trivadis
15
              NoSQL and SOA
              10.10.2012
NoSQL Database Types
              Key/Value             Column Family      Document          Graph
 Key/Value Stores
Design            Collections of  Columns and          Key/Value pairs   Focus on the
    Ordered Key-Value Stores Colum Families.
                  Key/Value Pairs                      but value is      connections
                                  Accesses directly    interpreted by    between data and
    Big Table Stores (map-of-maps-of-maps)
                                  the column values.   the database      the fast navigation

   Document Stores
Scalability/ +++                   +++                ++                ++
Performance
  Graph Databases
Aggregate-  Yes                    Yes                Yes               No
oriented
Complexity         +                ++                 ++                +++
Inspiration and    Berkley DB,      SAP Sybase IQ,     Lotus Notes       Graph Theory
Relation           Memcached,       BigTable
                   Distributed
                   Hashmaps

NoSQL              Voldemort        Hbase              CouchDB           Sones
Products           Redis            Cassandra          MongoDB           Neo4J
                   Riak             Hypertable         OrientDB          InfoGrid
                                    Amazon SimpleDB    RavenDB           FlockDB
                  2012 © Trivadis
16
                  NoSQL and SOA
                  10.10.2012
NoSQL Database Types
Size




       Key-value stores


                                  Column Family

                                                  Document
                                                             Graph




                Relational



                                                              Complexity

                2012 © Trivadis
17
                NoSQL and SOA
                10.10.2012
Key Value Databases

 A key-value store is a simple hash table
 Primarily used when all access to the database is via primary key
 Simplest NoSQL data stores to use (from an API perspective)
            PUT, GET, DELETE (matches REST)

 Value is a blob with the data store not caring or knowing what is inside
 Aggregate-Oriented


Suitable Use Cases
•        Storing Session Information
•        User Profiles, Preferences
•        Shopping Cart Data


                        2012 © Trivadis
    18
                        NoSQL and SOA
                        10.10.2012
Column-Family Stores

 Store data in column families as rows that have many columns associated with a
  row key
 Column families are groups of related data, often accessed together
 Aggregate-Oriented




Suitable Use Cases
•        Event Logging
•        Content Management Systems
•        Counters
                                           Source: NoSQL Distilled
•        Expiring Usage
                         2012 © Trivadis
    19
                         NoSQL and SOA
                         10.10.2012
Document Databases

 Documents are the main concept
 Stores and retrieves documents, which can be XML, JSON, BSON, …
 Documents are self-describing, hierarchical tree data structures which can consist
  of maps, collections and scalar values
 Documents stored are similar to each other but do not have to be exactly the
  same
 Aggregate-Oriented

Suitable Use Cases
•        Event Logging
•        Content Management Systems
•        Web Analytics or Real-Time Analytics
•        Product Catalog
                         2012 © Trivadis
    20
                         NoSQL and SOA
                         10.10.2012
Document Database - MongoDB




         2012 © Trivadis
21
         NoSQL and SOA
         10.10.2012
Graph Databases

 Allow to store entities and relationships between these entities
 Entities are known as nodes, which have properties
 Relations are known as edges, which also have properties
 A query on the graph is also known as traversing the graph
 Traversing the relationships is very fast

                                                                                                            Tag
                                                                              Customer


Suitable Use Cases
                                                Country
                                                                                         RATED            TAG



•        Connected Data
                                                                    ADDRESS
                                                   COUNTRY                                            Product


•        Routing, Dispatch and Location-Based                                 BILLING_
                                                                                          LINE_ITEM
         Services
                                                                              ADDRESS
                                                          Address


         Recommendation Engines
                                                                                         Order
•                                                                       DELIVERY_
                                                                        ADDRESS




                      2012 © Trivadis
    22
                      NoSQL and SOA
                      10.10.2012
Graph Database – Neo4J




Query through Cypher
 START MATCH WHERE RETURN ORDER BY LIMIT
 customer=node:Customer(email = "david@dmband.com")
 customer-[:ORDERED]->order-[item:LINEITEM]->product
 order.date > 20120101
 product.name, sum(item.amount) AS product
 products DESC 20

                     2012 © Trivadis
 23
                     NoSQL and SOA
                     10.10.2012
Agenda


1. Why NoSQL and what is it?
2. NoSQL Database Types
3. Polyglot Persistence
4. NoSQL and Service-Oriented Architecture
5. Summary




             2012 © Trivadis
24
             NoSQL and SOA
             10.10.2012
Polyglot Persistence


In 2006, Neal Ford coined the term Polyglot Programming
 Applications should be written in a mix of languages to take advantage
  of the fact that different languages are suitable for tackling different
  problems
Polyglot Persistence defines a a hybrid approach to persistence
 Using multiple data storage technologies
 Selected based on the way data is being used by individual applications
      Why store binary images in relational databases, when there are better
       storage systems?
 Can occur both over the enterprise as well as within a single application



                 2012 © Trivadis
25
                 NoSQL and SOA
                 10.10.2012
„Traditional“ Persistence Model
Polyglot Persistence                                                          E-commerce Application




Today we use the same
database for all kind of data         Shopping cart data      User Sessions        Completed Order     Product Catalog   Recomendations




•    Business transactions, session
     management                                                                      RDBMS


     data, reporting, logging
     information, content
     information, ...
                                                                   Polygot Persistence Model
No need for same properties of
availability, consistency or
                                                                              E-commerce Application




backup requirements
Polyglot Data Storage Usage           Shopping cart data      User Sessions        Completed Order     Product Catalog   Recomendations




allows to mix and match
Relational and NoSQL data
stores                                                 Key-Value                     RDMBS              Document            Graph




                 2012 © Trivadis
26
                 NoSQL and SOA
                 10.10.2012
Polyglot Persistence – Challenges

• Decisions
         • Have to decide what data storage technology to use
         • Today it„s easier to go with relational
• New Data Access APIs
     •     Each data store has its own mechanisms for accessing the data
     •     Different API‟s
                                                                    Service-Oriented Polygot Persistence Model
                                                                                         E-commerce Application

     •     Solution: Wrap the data access
           code into services (Data/Entity
           Service) exposed to applications
     •     Will enforce a contract/schema
                                               Shopping cart data        User Sessions        Completed Order      Product Catalog    Recomendations




           to a schemaless database
                                                 Key-Value                                                                               Graph
                                                                                                RDMBS               Document

                                               Shopping Cart            User Session                              Product Catalog    Recomendation
                                                 Service                  Service            Order Service            Service           Service




                     2012 © Trivadis
27
                     NoSQL and SOA
                     10.10.2012
Polyglot Persistence – Challenges


•    Immaturity
     • NoSQL tools are still young, full of rough edges that new tools have
     • Not much experience, we don„t know how to use them well
     • No patterns and best practices exist yet


•    Organizational Change
     • How will the different data groups in an enterprise react to this new
       technology


•    Dealing with eventual consistency paradigm
     • Reaction of different stakeholders to the fact that data could be stale
     • How to enforce rules to sync data across systems

                  2012 © Trivadis
28
                  NoSQL and SOA
                  10.10.2012
Agenda


1. What is NoSQL and Big Data
2. NoSQL Database Types
3. Polyglot Persistence
4. NoSQL and Service-Oriented Architecture
5. Summary




              2012 © Trivadis
29
              NoSQL and SOA
              10.10.2012
Data Access Architecture for Polyglot Persistence


well known design patterns are still valid!
some best practices we know in data access are still valid!

     Consumer                            Consumer       Consumer


           REST/SOAP                                          REST/SOAP



     Service                                            Service

                                                 REST
     Repository/DAO                                     Repository/DAO


     O/R Mapping

                                                                   ???
               SQL

                                         REST API

     RDBMS                               NoSQL          NoSQL



                       2012 © Trivadis
30
                       NoSQL and SOA
                       10.10.2012
Middle Tier Architecture for Polyglot Persistence




                                                                                                                                                  Resource Tier
                                                                                 Middle Tier
     Consumer                   Integration           Service                   Application                        Domain           Integration

                                                                                                              Domain Service Bean




                                                        Web Service Exporter




                                                                                   Application Service Bean
                                   REST
        Composite Application




                                                                                                                Factory Bean

                                  SOAP
                                                                                                                                    O/R Mapping
                                                                                                                Domain Objects
                                                                                                                                    NoSQL API
                                                                                                                 Repository Bean
                                                                                                                    Aggregate        SQL API


                                                                                                                  DAO Bean


                                                                               Data Transfer Objects (DTO)




                                    2012 © Trivadis
31
                                    NoSQL and SOA
                                    10.10.2012
Polyglot Persistence with Spring Data


makes it easier to build Spring-powered applications that use new data
access technologies
provide improved support for relational database technologies
Commons project supports Polyglot Persistence
Currently support for:
•    JPA and JDBC (relational)                                  Consumer


•    Apache Hadoop                                                    REST/SOAP



•    GemFire                                                    Service


•    REST                                                       Repository/DAO


•    Redis
•    MongoDB                                                               ???



•    Neo4J
•    Hbase                                                      NoSQL




                 2012 © Trivadis
32
                 NoSQL and SOA
                 10.10.2012
Spring Data – Mapping to Relational Database (using JPA)
                                   Annotations define the mapping:
                                   @Entity, @Id, @Column,
                                   @OneToOne, @OneToMany, @JoinCol
                                   umn,


                                                        Consumer


                                                              REST/SOAP



                                                        Service


                                                        Repository/DAO




                                                                   ???




                                                        NoSQL




           2012 © Trivadis
33
           NoSQL and SOA
           10.10.2012
Spring Data – Mapping to Relational Database
                                                                                                          Consumer


                                                                                                                REST/SOAP



                                                                                                          Service


public interface CustomerRepository extends Repository<Customer, Long> {                                  Repository/DAO


   Customer findByEmailAddress(EmailAddress emailAddress);
}                                                                                                                    ???




@Repository                                                                                               NoSQL


@Profile(“jpa")
class JpaCustomerRepository implements CustomerRepository {
   @Override
   public Customer findByEmailAddress(EmailAddress emailAddress) {

     TypedQuery<Customer> query = em.createQuery(
               "select c from Customer c where c.emailAddress = :email“, Customer.class);
     query.setParameter("email", emailAddress);
                                                              Customer guido= repository.findByEmailAddress(new
     return query.getSingleResult();                          EmailAddress(“guido@hotmail.com"));
}
                                                                 Customer anotherCust= new Customer(“Peter", “Sample");
<jpa:repositories base-package="com.oreilly.springdata.jpa" />   anotherCust.setEmailAddress(guido.getEmailAddress());

                                                            repository.save(anotherCust);
<bean class="org.springframework.orm.jpa.JpaTransactionManager">
    <property name="entityManagerFactory" ref="entityManagerFactory" />
</bean>

<bean id="entityManagerFactory" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
   <property name="dataSource" ref="dataSource" />
   <property name="packagesToScan" value="com.oreilly.springdata.jpa" />
</bean>

                      2012 © Trivadis
34
                      NoSQL and SOA
                      10.10.2012
Spring Data – Mapping to MongoDB
                               Annotations define the mapping:
                               @Document, @Id, @Indexed,
                               @PersistenceConstructor, @Compound
                               Index, @DBRef, @GeoSpatialIndex, @V
                               alue


                                                     Consumer


                                                           REST/SOAP



                                                     Service


                                                     Repository/DAO




                                                                ???




                                                     NoSQL




          2012 © Trivadis
35
          NoSQL and SOA
          10.10.2012
Spring Data – Generic Repositories for MongoDB
                                                                                                    Consumer


                                                                                                          REST/SOAP



                                                                                                    Service


public interface CustomerRepository extends Repository<Customer, Long> {                            Repository/DAO


   Customer findByEmailAddress(EmailAddress emailAddress);
}                                                                                                              ???




@Repository                                                                                         NoSQL


@Profile("mongodb")
class MongoDbCustomerRepository implements CustomerRepository {
   @Override
   public Customer findByEmailAddress(EmailAddress emailAddress) {

            Query query = query(where("emailAddress").is(emailAddress));
            return operations.findOne(query, Customer.class);
     }

<mongo:db-factory id="mongoDbFactory" dbname="e-store" />

<mongo:mapping-converter id="mongoConverter" base-package="com.oreilly.springdata.mongodb">
<mongo:custom-converters base-package="com.oreilly.springdata.mongodb" />
</mongo:mapping-converter>

<bean id="mongoTemplate" class="org.springframework.data.mongodb.core.MongoTemplate">
                                                          Customer guido= repository.findByEmailAddress(new
 <constructor-arg ref="mongoDbFactory" />                 EmailAddress(“guido@hotmail.com"));
 <constructor-arg ref="mongoConverter" />
 <property name="writeConcern" value="SAFE" />            Customer anotherCust= new Customer(“Peter", “Sample");
</bean>                                                   anotherCust.setEmailAddress(guido.getEmailAddress());
<mongo:repositories base-package="com.oreilly.springdata.mongodb" />
                                                           repository.save(anotherCust);
                     2012 © Trivadis
36
                     NoSQL and SOA
                     10.10.2012
Spring Data – Mapping to Neo4J
                                                                       Annotations define the mapping:
                                                                       @NodeEntity, RelationShipEntity, @Gra
                                                                       phId, @RelatedTo, @RelatedToVia, @E
                                                                       ndNode, @Fetch,




                                                                 Tag
                               Customer
 Country
                                              RATED            TAG


                     ADDRESS
     COUNTRY                                               Product


                               BILLING_
                               ADDRESS         LINE_ITEM
           Address

                                              Order
                         DELIVERY_
                         ADDRESS


                                          2012 © Trivadis
37
                                          NoSQL and SOA
                                          10.10.2012
Spring Data – Generic Repositories for MongoDB
                                                                                                      Consumer


                                                                                                            REST/SOAP



                                                                                                      Service


                                                                                                      Repository/DAO




public interface CustomerRepository extends GraphRepository<Customer> {
                                                                                                                 ???




   Customer findByEmailAddress(EmailAddress emailAddress);
}
                                                                                                      NoSQL




<neo4j:config graphDatabaseService="graphDatabaseService" />
<neo4j:repositories base-package="com.oreilly.springdata.neo4j" />

<bean id="graphDatabaseService" class="org.neo4j.kernel.EmbeddedGraphDatabase" destroy-method="shutdown">
  <constructor-arg value="target/graph.db" />
</bean>



Customer guido= repository.findByEmailAddress(new
EmailAddress(“guido@hotmail.com"));

Customer anotherCust= new Customer(“Peter", “Sample");
anotherCust.setEmailAddress(guido.getEmailAddress());

repository.save(anotherCust);




                      2012 © Trivadis
38
                      NoSQL and SOA
                      10.10.2012
Expose contract-first Web service
                                                          Consumer


                                                                REST/SOAP



                                                          Service


                                                          Repository/DAO


Use any Java Web Service Framework which supports
Contract-First approach                                              ???




Can be SOAP or can be REST                                NoSQL



Maps the data contract (WSDL or WADL) to the schemaless
database
Uses the different Repository implementations
Must handle data migration issues together with the
Repository




              2012 © Trivadis
39
              NoSQL and SOA
              10.10.2012
Using SOA for Integrating “old” with “new” world




           2012 © Trivadis

           Five Cool Use Cases for the Spring Component of Oracle SOA Suite
           1.10.2012
NoSQL databases as
Caching in a SOA

Results are returned from cache rather than invoking always
the external service
 Product data is rather static, so ideal candidate for caching

    Proxy                       Business                              1   Product DB
   Service                      Service
                            2                  3

          Result
         Cache
Oracle Service
Bus




                   2012 © Trivadis

                   Effective Fault Handling in Oracle SOA Suite 11g
                   12.10.2012
NoSQL databases as
Caching in a SOA

Using Mongo DB to cache results in a SOA




             2012 © Trivadis
42
             NoSQL and SOA
             10.10.2012
Schemaless – We still have to migrate the data!


With RDMBS we are used to keep DDL scripts together           Customer


with DML scripts for each single data model change
                                                              Name: Peter Sample
                                                              First Name: Peter
                                                              Last Name: Sample
                                                               BillingAddress
                                                               Billing Address

     •   Has to be in sync with the data access code            Street: Somestreet 10   Version 1.0
                                                                City: Somewhere
                                                                Postal Code:55901
                                                                PostalCode: 55901

RDBMS has to be changed before the application
is changed => possible application downtime                   Customer
     •   This is what the schemaless approach of most NoSQL   Name: Peter Sample
                                                              FirstName: Peter
         DB tries to avoid                                    LastName: Sample
                                                                                        Transition
                                                               Billing Address
                                                                                        Version 1.0 => 2.0

Schemaless DBs still need careful migration, due to
                                                                Street: Somestreet 10
                                                                City: Somewhere

implicit schema in any data access code
                                                                PostalCode: 55901




But a more “on-demand” approach is possible                   Customer

     •   Code can read data in a way that it tolerant to      First Name: Peter
                                                              Last Name: Sample

         changes in the data‟s implicit schema and migrate     Billing Address          Version 2.0

         the data on the next update
                                                                Street: Somestreet 10
                                                                City: Somewhere
                                                                PostalCode: 55901

     •   Similar to service versioning => gradual change


                    2012 © Trivadis
43
                    NoSQL and SOA
                    10.10.2012
Agenda


1. What is NoSQL and Big Data
2. NoSQL Database Types
3. Polyglot Persistence
4. NoSQL and Service-Oriented Architecture
5. Summary




              2012 © Trivadis
44
              NoSQL and SOA
              10.10.2012
Pros & Cons of NoSQL compared to RDBMS


Pros                              Cons
• No O/R impedance mismatch       • Lacks in tool and framework
                                    support
• Can easily evolve schemas
                                  • Few other implementations =>
• Can represent semi-structured
                                    potential lock in
  info
                                  • No support for ad-hoc queries
• Can represent graphs/networks
  (with performance)              • Another/A new database in
                                    production to take care of




             2012 © Trivadis
45
             NoSQL and SOA
             10.10.2012
Summary


Relational databases are here to stay but NoSQL offers new persistence
model
Polyglot Persistence will be the future
Schemaless does not mean there is no data migration! => but a more on-
demand model might be possible
Encapsulate data access code to be able to switch databases
Service-orientation provides the data contract to a NoSQL database => to
make information reusable
Don„t commit to a NoSQL until you have done a significant PoC
Make sure that Operations people (DBAs) are on board early enough
Non-relational is not new in an enterprise (OLTP vs. OLAP)

              2012 © Trivadis
46
              NoSQL and SOA
              10.10.2012
Further Information




           2012 © Trivadis
47
           NoSQL and SOA
           10.10.2012
VIELEN DANK.
                                                                       Trivadis

                                                                       Guido Schmutz
                                                                       guido.schmutz@trivadis.com

                                                                       info@trivadis.com
                                                                       www.trivadis.com




BASEL    BERN   LAUSANNE     ZÜRICH      DÜSSELDORF   FRANKFURT A.M.   FREIBURG I.BR.   HAMBURG   MÜNCHEN   STUTTGART   WIEN




                       2012 © Trivadis
 48
                       NoSQL and SOA
                       10.10.2012
Possible Use Cases

 NoSQL for parallel ETL?
 NoSQL for modern BI
 NoSQL for stateful Middletier (i.e. shopping cart)
 NoSQL for aggregated master data (i.e. through REST for Web apps)
 NoSQL for a CMS-Store, directly accessible through REST
 NoSQL as a local Store for Mobile applications
 NoSQL for Event Sourcing and CQRS architectures




              2012 © Trivadis
49
              NoSQL and SOA
              10.10.2012

More Related Content

Viewers also liked

How_to_build_GameServer_2
How_to_build_GameServer_2How_to_build_GameServer_2
How_to_build_GameServer_2Peter Rybar
 
MongoDB performance tuning and load testing, NOSQL Now! 2013 Conference prese...
MongoDB performance tuning and load testing, NOSQL Now! 2013 Conference prese...MongoDB performance tuning and load testing, NOSQL Now! 2013 Conference prese...
MongoDB performance tuning and load testing, NOSQL Now! 2013 Conference prese...ronwarshawsky
 
Review Oracle OpenWorld 2015 - Overview, Main themes, Announcements and Future
Review Oracle OpenWorld 2015 - Overview, Main themes, Announcements and FutureReview Oracle OpenWorld 2015 - Overview, Main themes, Announcements and Future
Review Oracle OpenWorld 2015 - Overview, Main themes, Announcements and FutureLucas Jellema
 
The True State of the Oracle Public Cloud - Dutch Oracle Architects Platform ...
The True State of the Oracle Public Cloud - Dutch Oracle Architects Platform ...The True State of the Oracle Public Cloud - Dutch Oracle Architects Platform ...
The True State of the Oracle Public Cloud - Dutch Oracle Architects Platform ...Lucas Jellema
 
Systems on the edge - your stepping stones into Oracle Public PaaS Cloud - AM...
Systems on the edge - your stepping stones into Oracle Public PaaS Cloud - AM...Systems on the edge - your stepping stones into Oracle Public PaaS Cloud - AM...
Systems on the edge - your stepping stones into Oracle Public PaaS Cloud - AM...Lucas Jellema
 
10 Key MongoDB Performance Indicators
10 Key MongoDB Performance Indicators  10 Key MongoDB Performance Indicators
10 Key MongoDB Performance Indicators iammutex
 
Mobile Database and Service Oriented Architecture
Mobile Database and Service Oriented ArchitectureMobile Database and Service Oriented Architecture
Mobile Database and Service Oriented ArchitectureLucas Jellema
 
Introducing Node.js in an Oracle technology environment (including hands-on)
Introducing Node.js in an Oracle technology environment (including hands-on)Introducing Node.js in an Oracle technology environment (including hands-on)
Introducing Node.js in an Oracle technology environment (including hands-on)Lucas Jellema
 
Introducing Oracle Real-Time Integration Business Insight
Introducing Oracle Real-Time Integration Business InsightIntroducing Oracle Real-Time Integration Business Insight
Introducing Oracle Real-Time Integration Business InsightLucas Jellema
 
Gluent New World #02 - SQL-on-Hadoop : A bit of History, Current State-of-the...
Gluent New World #02 - SQL-on-Hadoop : A bit of History, Current State-of-the...Gluent New World #02 - SQL-on-Hadoop : A bit of History, Current State-of-the...
Gluent New World #02 - SQL-on-Hadoop : A bit of History, Current State-of-the...Mark Rittman
 
HMC_Industry_Report_Drone_Technology_160321[1]
HMC_Industry_Report_Drone_Technology_160321[1]HMC_Industry_Report_Drone_Technology_160321[1]
HMC_Industry_Report_Drone_Technology_160321[1]Robert Cheek
 
Oracle Database 12c - Introducing SQL Pattern Recognition through MATCH_RECOG...
Oracle Database 12c - Introducing SQL Pattern Recognition through MATCH_RECOG...Oracle Database 12c - Introducing SQL Pattern Recognition through MATCH_RECOG...
Oracle Database 12c - Introducing SQL Pattern Recognition through MATCH_RECOG...Lucas Jellema
 
What is the Oracle PaaS Cloud for Developers (Oracle Cloud Day, The Netherlan...
What is the Oracle PaaS Cloud for Developers (Oracle Cloud Day, The Netherlan...What is the Oracle PaaS Cloud for Developers (Oracle Cloud Day, The Netherlan...
What is the Oracle PaaS Cloud for Developers (Oracle Cloud Day, The Netherlan...Lucas Jellema
 
Fast querying indexing for performance (4)
Fast querying   indexing for performance (4)Fast querying   indexing for performance (4)
Fast querying indexing for performance (4)MongoDB
 
Ranges, ranges everywhere (Oracle SQL)
Ranges, ranges everywhere (Oracle SQL)Ranges, ranges everywhere (Oracle SQL)
Ranges, ranges everywhere (Oracle SQL)Stew Ashton
 
How to monitor MongoDB
How to monitor MongoDBHow to monitor MongoDB
How to monitor MongoDBServer Density
 
Oracle Management Cloud - introduction, overview and getting started (AMIS, 2...
Oracle Management Cloud - introduction, overview and getting started (AMIS, 2...Oracle Management Cloud - introduction, overview and getting started (AMIS, 2...
Oracle Management Cloud - introduction, overview and getting started (AMIS, 2...Lucas Jellema
 

Viewers also liked (20)

How_to_build_GameServer_2
How_to_build_GameServer_2How_to_build_GameServer_2
How_to_build_GameServer_2
 
MongoDB performance tuning and load testing, NOSQL Now! 2013 Conference prese...
MongoDB performance tuning and load testing, NOSQL Now! 2013 Conference prese...MongoDB performance tuning and load testing, NOSQL Now! 2013 Conference prese...
MongoDB performance tuning and load testing, NOSQL Now! 2013 Conference prese...
 
Review Oracle OpenWorld 2015 - Overview, Main themes, Announcements and Future
Review Oracle OpenWorld 2015 - Overview, Main themes, Announcements and FutureReview Oracle OpenWorld 2015 - Overview, Main themes, Announcements and Future
Review Oracle OpenWorld 2015 - Overview, Main themes, Announcements and Future
 
The True State of the Oracle Public Cloud - Dutch Oracle Architects Platform ...
The True State of the Oracle Public Cloud - Dutch Oracle Architects Platform ...The True State of the Oracle Public Cloud - Dutch Oracle Architects Platform ...
The True State of the Oracle Public Cloud - Dutch Oracle Architects Platform ...
 
Systems on the edge - your stepping stones into Oracle Public PaaS Cloud - AM...
Systems on the edge - your stepping stones into Oracle Public PaaS Cloud - AM...Systems on the edge - your stepping stones into Oracle Public PaaS Cloud - AM...
Systems on the edge - your stepping stones into Oracle Public PaaS Cloud - AM...
 
AMIS Oracle OpenWorld 2013 Review Part 1 - Intro Overview Innovation, Hardwar...
AMIS Oracle OpenWorld 2013 Review Part 1 - Intro Overview Innovation, Hardwar...AMIS Oracle OpenWorld 2013 Review Part 1 - Intro Overview Innovation, Hardwar...
AMIS Oracle OpenWorld 2013 Review Part 1 - Intro Overview Innovation, Hardwar...
 
10 Key MongoDB Performance Indicators
10 Key MongoDB Performance Indicators  10 Key MongoDB Performance Indicators
10 Key MongoDB Performance Indicators
 
Mobile Database and Service Oriented Architecture
Mobile Database and Service Oriented ArchitectureMobile Database and Service Oriented Architecture
Mobile Database and Service Oriented Architecture
 
Introducing Node.js in an Oracle technology environment (including hands-on)
Introducing Node.js in an Oracle technology environment (including hands-on)Introducing Node.js in an Oracle technology environment (including hands-on)
Introducing Node.js in an Oracle technology environment (including hands-on)
 
Introducing Oracle Real-Time Integration Business Insight
Introducing Oracle Real-Time Integration Business InsightIntroducing Oracle Real-Time Integration Business Insight
Introducing Oracle Real-Time Integration Business Insight
 
Gluent New World #02 - SQL-on-Hadoop : A bit of History, Current State-of-the...
Gluent New World #02 - SQL-on-Hadoop : A bit of History, Current State-of-the...Gluent New World #02 - SQL-on-Hadoop : A bit of History, Current State-of-the...
Gluent New World #02 - SQL-on-Hadoop : A bit of History, Current State-of-the...
 
HMC_Industry_Report_Drone_Technology_160321[1]
HMC_Industry_Report_Drone_Technology_160321[1]HMC_Industry_Report_Drone_Technology_160321[1]
HMC_Industry_Report_Drone_Technology_160321[1]
 
Oracle Database 12c - Introducing SQL Pattern Recognition through MATCH_RECOG...
Oracle Database 12c - Introducing SQL Pattern Recognition through MATCH_RECOG...Oracle Database 12c - Introducing SQL Pattern Recognition through MATCH_RECOG...
Oracle Database 12c - Introducing SQL Pattern Recognition through MATCH_RECOG...
 
What is the Oracle PaaS Cloud for Developers (Oracle Cloud Day, The Netherlan...
What is the Oracle PaaS Cloud for Developers (Oracle Cloud Day, The Netherlan...What is the Oracle PaaS Cloud for Developers (Oracle Cloud Day, The Netherlan...
What is the Oracle PaaS Cloud for Developers (Oracle Cloud Day, The Netherlan...
 
Fast querying indexing for performance (4)
Fast querying   indexing for performance (4)Fast querying   indexing for performance (4)
Fast querying indexing for performance (4)
 
SQL Outer Joins for Fun and Profit
SQL Outer Joins for Fun and ProfitSQL Outer Joins for Fun and Profit
SQL Outer Joins for Fun and Profit
 
Ranges, ranges everywhere (Oracle SQL)
Ranges, ranges everywhere (Oracle SQL)Ranges, ranges everywhere (Oracle SQL)
Ranges, ranges everywhere (Oracle SQL)
 
How to monitor MongoDB
How to monitor MongoDBHow to monitor MongoDB
How to monitor MongoDB
 
Oracle SaaS Applications Overview
Oracle SaaS Applications OverviewOracle SaaS Applications Overview
Oracle SaaS Applications Overview
 
Oracle Management Cloud - introduction, overview and getting started (AMIS, 2...
Oracle Management Cloud - introduction, overview and getting started (AMIS, 2...Oracle Management Cloud - introduction, overview and getting started (AMIS, 2...
Oracle Management Cloud - introduction, overview and getting started (AMIS, 2...
 

Similar to NoSQL and SOA

Getting started with entity framework revised 9 09
Getting started with entity framework revised 9 09Getting started with entity framework revised 9 09
Getting started with entity framework revised 9 09manisoft84
 
Sql azure database under the hood
Sql azure database under the hoodSql azure database under the hood
Sql azure database under the hoodguest2dd056
 
Sql azure database under the hood
Sql azure database under the hoodSql azure database under the hood
Sql azure database under the hoodEduardo Castro
 
Drupal + HTML5 + CSS3 + JS = Rich Internet Application
Drupal + HTML5 + CSS3 + JS = Rich Internet ApplicationDrupal + HTML5 + CSS3 + JS = Rich Internet Application
Drupal + HTML5 + CSS3 + JS = Rich Internet ApplicationAppnovation Technologies
 
Drupal + HTML5 + CSS3 + JS = Rich Internet Application
Drupal + HTML5 + CSS3 + JS = Rich Internet ApplicationDrupal + HTML5 + CSS3 + JS = Rich Internet Application
Drupal + HTML5 + CSS3 + JS = Rich Internet ApplicationDrupalcampAtlanta2012
 
3. Sql Services 概览
3. Sql Services 概览3. Sql Services 概览
3. Sql Services 概览GaryYoung
 
SQL Data Service Overview
SQL Data Service OverviewSQL Data Service Overview
SQL Data Service OverviewEric Nelson
 
REDIculouS: Introduction to Redis
REDIculouS: Introduction to RedisREDIculouS: Introduction to Redis
REDIculouS: Introduction to Redispootsbook
 
The Modern Web, Part 1: Mobility
The Modern Web, Part 1: MobilityThe Modern Web, Part 1: Mobility
The Modern Web, Part 1: MobilityDavid Pallmann
 
Ebs architecture con9036_pdf_9036_0001
Ebs architecture con9036_pdf_9036_0001Ebs architecture con9036_pdf_9036_0001
Ebs architecture con9036_pdf_9036_0001jucaab
 
Sql azure data services OData
Sql azure data services ODataSql azure data services OData
Sql azure data services ODataEduardo Castro
 
SQL Azure Federation and Scalability
SQL Azure Federation and ScalabilitySQL Azure Federation and Scalability
SQL Azure Federation and ScalabilityEduardo Castro
 
When worlds Collide: HTML5 Meets the Cloud
When worlds Collide: HTML5 Meets the CloudWhen worlds Collide: HTML5 Meets the Cloud
When worlds Collide: HTML5 Meets the CloudDavid Pallmann
 
The Modern Web Part 4: Cloud Computing
The Modern Web Part 4: Cloud ComputingThe Modern Web Part 4: Cloud Computing
The Modern Web Part 4: Cloud ComputingDavid Pallmann
 
A Guide to the SOA Galaxy: Strategy, Design and Best Practices
A Guide to the SOA Galaxy: Strategy, Design and Best PracticesA Guide to the SOA Galaxy: Strategy, Design and Best Practices
A Guide to the SOA Galaxy: Strategy, Design and Best PracticesDmitri Shiryaev
 
Co-existence or competition - RDBMS and Hadoop
Co-existence or competition  - RDBMS and HadoopCo-existence or competition  - RDBMS and Hadoop
Co-existence or competition - RDBMS and HadoopFlytxt
 
Co existence or Competitions? RDBMS and Hadoop
Co existence or Competitions? RDBMS and HadoopCo existence or Competitions? RDBMS and Hadoop
Co existence or Competitions? RDBMS and HadoopFlytxt
 

Similar to NoSQL and SOA (20)

Getting started with entity framework revised 9 09
Getting started with entity framework revised 9 09Getting started with entity framework revised 9 09
Getting started with entity framework revised 9 09
 
Sql azure database under the hood
Sql azure database under the hoodSql azure database under the hood
Sql azure database under the hood
 
Sql azure database under the hood
Sql azure database under the hoodSql azure database under the hood
Sql azure database under the hood
 
Drupal + HTML5 + CSS3 + JS = Rich Internet Application
Drupal + HTML5 + CSS3 + JS = Rich Internet ApplicationDrupal + HTML5 + CSS3 + JS = Rich Internet Application
Drupal + HTML5 + CSS3 + JS = Rich Internet Application
 
Drupal + HTML5 + CSS3 + JS = Rich Internet Application
Drupal + HTML5 + CSS3 + JS = Rich Internet ApplicationDrupal + HTML5 + CSS3 + JS = Rich Internet Application
Drupal + HTML5 + CSS3 + JS = Rich Internet Application
 
3. Sql Services 概览
3. Sql Services 概览3. Sql Services 概览
3. Sql Services 概览
 
SQL Data Service Overview
SQL Data Service OverviewSQL Data Service Overview
SQL Data Service Overview
 
REDIculouS: Introduction to Redis
REDIculouS: Introduction to RedisREDIculouS: Introduction to Redis
REDIculouS: Introduction to Redis
 
The Modern Web, Part 1: Mobility
The Modern Web, Part 1: MobilityThe Modern Web, Part 1: Mobility
The Modern Web, Part 1: Mobility
 
ADO.NET Data Services
ADO.NET Data ServicesADO.NET Data Services
ADO.NET Data Services
 
Ebs architecture con9036_pdf_9036_0001
Ebs architecture con9036_pdf_9036_0001Ebs architecture con9036_pdf_9036_0001
Ebs architecture con9036_pdf_9036_0001
 
Sql azure data services OData
Sql azure data services ODataSql azure data services OData
Sql azure data services OData
 
SQL Azure Federation and Scalability
SQL Azure Federation and ScalabilitySQL Azure Federation and Scalability
SQL Azure Federation and Scalability
 
When worlds Collide: HTML5 Meets the Cloud
When worlds Collide: HTML5 Meets the CloudWhen worlds Collide: HTML5 Meets the Cloud
When worlds Collide: HTML5 Meets the Cloud
 
Business Models for Interoperability
Business Models for InteroperabilityBusiness Models for Interoperability
Business Models for Interoperability
 
The Modern Web Part 4: Cloud Computing
The Modern Web Part 4: Cloud ComputingThe Modern Web Part 4: Cloud Computing
The Modern Web Part 4: Cloud Computing
 
Power pointshow
Power pointshowPower pointshow
Power pointshow
 
A Guide to the SOA Galaxy: Strategy, Design and Best Practices
A Guide to the SOA Galaxy: Strategy, Design and Best PracticesA Guide to the SOA Galaxy: Strategy, Design and Best Practices
A Guide to the SOA Galaxy: Strategy, Design and Best Practices
 
Co-existence or competition - RDBMS and Hadoop
Co-existence or competition  - RDBMS and HadoopCo-existence or competition  - RDBMS and Hadoop
Co-existence or competition - RDBMS and Hadoop
 
Co existence or Competitions? RDBMS and Hadoop
Co existence or Competitions? RDBMS and HadoopCo existence or Competitions? RDBMS and Hadoop
Co existence or Competitions? RDBMS and Hadoop
 

More from Guido Schmutz

30 Minutes to the Analytics Platform with Infrastructure as Code
30 Minutes to the Analytics Platform with Infrastructure as Code30 Minutes to the Analytics Platform with Infrastructure as Code
30 Minutes to the Analytics Platform with Infrastructure as CodeGuido Schmutz
 
Event Broker (Kafka) in a Modern Data Architecture
Event Broker (Kafka) in a Modern Data ArchitectureEvent Broker (Kafka) in a Modern Data Architecture
Event Broker (Kafka) in a Modern Data ArchitectureGuido Schmutz
 
Big Data, Data Lake, Fast Data - Dataserialiation-Formats
Big Data, Data Lake, Fast Data - Dataserialiation-FormatsBig Data, Data Lake, Fast Data - Dataserialiation-Formats
Big Data, Data Lake, Fast Data - Dataserialiation-FormatsGuido Schmutz
 
ksqlDB - Stream Processing simplified!
ksqlDB - Stream Processing simplified!ksqlDB - Stream Processing simplified!
ksqlDB - Stream Processing simplified!Guido Schmutz
 
Kafka as your Data Lake - is it Feasible?
Kafka as your Data Lake - is it Feasible?Kafka as your Data Lake - is it Feasible?
Kafka as your Data Lake - is it Feasible?Guido Schmutz
 
Event Hub (i.e. Kafka) in Modern Data Architecture
Event Hub (i.e. Kafka) in Modern Data ArchitectureEvent Hub (i.e. Kafka) in Modern Data Architecture
Event Hub (i.e. Kafka) in Modern Data ArchitectureGuido Schmutz
 
Solutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache KafkaSolutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache KafkaGuido Schmutz
 
Event Hub (i.e. Kafka) in Modern Data (Analytics) Architecture
Event Hub (i.e. Kafka) in Modern Data (Analytics) ArchitectureEvent Hub (i.e. Kafka) in Modern Data (Analytics) Architecture
Event Hub (i.e. Kafka) in Modern Data (Analytics) ArchitectureGuido Schmutz
 
Building Event Driven (Micro)services with Apache Kafka
Building Event Driven (Micro)services with Apache KafkaBuilding Event Driven (Micro)services with Apache Kafka
Building Event Driven (Micro)services with Apache KafkaGuido Schmutz
 
Location Analytics - Real-Time Geofencing using Apache Kafka
Location Analytics - Real-Time Geofencing using Apache KafkaLocation Analytics - Real-Time Geofencing using Apache Kafka
Location Analytics - Real-Time Geofencing using Apache KafkaGuido Schmutz
 
Solutions for bi-directional integration between Oracle RDBMS and Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS and Apache KafkaSolutions for bi-directional integration between Oracle RDBMS and Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS and Apache KafkaGuido Schmutz
 
What is Apache Kafka? Why is it so popular? Should I use it?
What is Apache Kafka? Why is it so popular? Should I use it?What is Apache Kafka? Why is it so popular? Should I use it?
What is Apache Kafka? Why is it so popular? Should I use it?Guido Schmutz
 
Solutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache KafkaSolutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache KafkaGuido Schmutz
 
Location Analytics Real-Time Geofencing using Kafka
Location Analytics Real-Time Geofencing using KafkaLocation Analytics Real-Time Geofencing using Kafka
Location Analytics Real-Time Geofencing using KafkaGuido Schmutz
 
Streaming Visualisation
Streaming VisualisationStreaming Visualisation
Streaming VisualisationGuido Schmutz
 
Kafka as an event store - is it good enough?
Kafka as an event store - is it good enough?Kafka as an event store - is it good enough?
Kafka as an event store - is it good enough?Guido Schmutz
 
Solutions for bi-directional Integration between Oracle RDMBS & Apache Kafka
Solutions for bi-directional Integration between Oracle RDMBS & Apache KafkaSolutions for bi-directional Integration between Oracle RDMBS & Apache Kafka
Solutions for bi-directional Integration between Oracle RDMBS & Apache KafkaGuido Schmutz
 
Fundamentals Big Data and AI Architecture
Fundamentals Big Data and AI ArchitectureFundamentals Big Data and AI Architecture
Fundamentals Big Data and AI ArchitectureGuido Schmutz
 
Location Analytics - Real-Time Geofencing using Kafka
Location Analytics - Real-Time Geofencing using Kafka Location Analytics - Real-Time Geofencing using Kafka
Location Analytics - Real-Time Geofencing using Kafka Guido Schmutz
 
Streaming Visualization
Streaming VisualizationStreaming Visualization
Streaming VisualizationGuido Schmutz
 

More from Guido Schmutz (20)

30 Minutes to the Analytics Platform with Infrastructure as Code
30 Minutes to the Analytics Platform with Infrastructure as Code30 Minutes to the Analytics Platform with Infrastructure as Code
30 Minutes to the Analytics Platform with Infrastructure as Code
 
Event Broker (Kafka) in a Modern Data Architecture
Event Broker (Kafka) in a Modern Data ArchitectureEvent Broker (Kafka) in a Modern Data Architecture
Event Broker (Kafka) in a Modern Data Architecture
 
Big Data, Data Lake, Fast Data - Dataserialiation-Formats
Big Data, Data Lake, Fast Data - Dataserialiation-FormatsBig Data, Data Lake, Fast Data - Dataserialiation-Formats
Big Data, Data Lake, Fast Data - Dataserialiation-Formats
 
ksqlDB - Stream Processing simplified!
ksqlDB - Stream Processing simplified!ksqlDB - Stream Processing simplified!
ksqlDB - Stream Processing simplified!
 
Kafka as your Data Lake - is it Feasible?
Kafka as your Data Lake - is it Feasible?Kafka as your Data Lake - is it Feasible?
Kafka as your Data Lake - is it Feasible?
 
Event Hub (i.e. Kafka) in Modern Data Architecture
Event Hub (i.e. Kafka) in Modern Data ArchitectureEvent Hub (i.e. Kafka) in Modern Data Architecture
Event Hub (i.e. Kafka) in Modern Data Architecture
 
Solutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache KafkaSolutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache Kafka
 
Event Hub (i.e. Kafka) in Modern Data (Analytics) Architecture
Event Hub (i.e. Kafka) in Modern Data (Analytics) ArchitectureEvent Hub (i.e. Kafka) in Modern Data (Analytics) Architecture
Event Hub (i.e. Kafka) in Modern Data (Analytics) Architecture
 
Building Event Driven (Micro)services with Apache Kafka
Building Event Driven (Micro)services with Apache KafkaBuilding Event Driven (Micro)services with Apache Kafka
Building Event Driven (Micro)services with Apache Kafka
 
Location Analytics - Real-Time Geofencing using Apache Kafka
Location Analytics - Real-Time Geofencing using Apache KafkaLocation Analytics - Real-Time Geofencing using Apache Kafka
Location Analytics - Real-Time Geofencing using Apache Kafka
 
Solutions for bi-directional integration between Oracle RDBMS and Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS and Apache KafkaSolutions for bi-directional integration between Oracle RDBMS and Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS and Apache Kafka
 
What is Apache Kafka? Why is it so popular? Should I use it?
What is Apache Kafka? Why is it so popular? Should I use it?What is Apache Kafka? Why is it so popular? Should I use it?
What is Apache Kafka? Why is it so popular? Should I use it?
 
Solutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache KafkaSolutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache Kafka
 
Location Analytics Real-Time Geofencing using Kafka
Location Analytics Real-Time Geofencing using KafkaLocation Analytics Real-Time Geofencing using Kafka
Location Analytics Real-Time Geofencing using Kafka
 
Streaming Visualisation
Streaming VisualisationStreaming Visualisation
Streaming Visualisation
 
Kafka as an event store - is it good enough?
Kafka as an event store - is it good enough?Kafka as an event store - is it good enough?
Kafka as an event store - is it good enough?
 
Solutions for bi-directional Integration between Oracle RDMBS & Apache Kafka
Solutions for bi-directional Integration between Oracle RDMBS & Apache KafkaSolutions for bi-directional Integration between Oracle RDMBS & Apache Kafka
Solutions for bi-directional Integration between Oracle RDMBS & Apache Kafka
 
Fundamentals Big Data and AI Architecture
Fundamentals Big Data and AI ArchitectureFundamentals Big Data and AI Architecture
Fundamentals Big Data and AI Architecture
 
Location Analytics - Real-Time Geofencing using Kafka
Location Analytics - Real-Time Geofencing using Kafka Location Analytics - Real-Time Geofencing using Kafka
Location Analytics - Real-Time Geofencing using Kafka
 
Streaming Visualization
Streaming VisualizationStreaming Visualization
Streaming Visualization
 

Recently uploaded

[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality AssuranceInflectra
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfIngrid Airi González
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityIES VE
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsPixlogix Infotech
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsNathaniel Shimoni
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...AliaaTarek5
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersRaghuram Pandurangan
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Alkin Tezuysal
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfNeo4j
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxLoriGlavin3
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterMydbops
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 

Recently uploaded (20)

[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdf
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a reality
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directions
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information Developers
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdf
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL Router
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 

NoSQL and SOA

  • 1. Welcome NoSQL and SOA Guido Schmutz 10.10.2012 BASEL BERN LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN 2012 © Trivadis 1 NoSQL and SOA 10.10.2012
  • 2. Guido Schmutz • Working for Trivadis for more than 15 years • Oracle ACE Director for Fusion Middleware and SOA • Co-Author of different books • Consultant, Trainer Software Architect for Java, Oracle, SOA and EDA • Member of Trivadis Architecture Board • Technology Manager @ Trivadis • More than 20 years of software development experience • Contact: guido.schmutz@trivadis.com • Blog: http://guidoschmutz.wordpress.com • Twitter: gschmutz 2012 © Trivadis 2 NoSQL and SOA 10.10.2012
  • 3. Agenda 1. Why NoSQL and what is it? 2. NoSQL Database Types 3. Polyglot Persistence 4. NoSQL and Service-Oriented Architecture 5. Summary 2012 © Trivadis 3 NoSQL and SOA 10.10.2012
  • 4. History of Database 1960s File-based, Network (CODASYL) and Hierarchical Databases 1970s Relational Database 1980 SQL became the standard query language Early 1990 Object-Databases Late 1990 XML Databases 2004 NoSQL Databases 2012 © Trivadis 4 NoSQL and SOA 10.10.2012
  • 5. What„s wrong with Relational Databases ? They are great …. • SQL provides a rich, declarative query language • Database enforce referential integrity • ACID semantics • Well understood by developers, database administrators • Well supported by different languages, frameworks and tools • Hibernate, JPA, JDBC, iBATIS, Entity Framework • Well understood and accepted by operations people (DBAs) • Configuration • Monitoring • Backup and Recovery • Tuning • Design 2012 © Trivadis 5 NoSQL and SOA 10.10.2012
  • 6. Relational Databases are great ... But! ORDER Order ID: 1001 Problem: Complex Object graphs Order Date: 15.9.2012 Customer  Object/Relational impedance mismatch CUSTOMER First Name: Peter Last Name: Sample  Complicated to map rich domain model Billing Address Street: Somestreet 10 to relational schema City: Somewhere Postal Code: 55901 ADDRESS  Performance issues Line Items  Many rows in many tables Name Ipod Touch Quantity 1 Price 220.95 ORDER_LINES  Many joins Monster Beat 2 190.00  Eager vs. lazy loading Apple Mouse 1 69.90 Problem: Schema evolution  Adding attributes to an object => have to add columns to table  Expensive, if lots of data in that table - Holding locks on the tables for long time - Application downtime … 2012 © Trivadis 6 NoSQL and SOA 10.10.2012
  • 7. ORDER Order Relational Databases are great ... But! ID: 1001 Order Date: 15.9.2012 Customer CUSTOMER First Name: Peter Last Name: Sample Billing Address Street: Somestreet 10 City: Somewhere Postal Code: 55901 ADDRESS Line Items Name Quantity Price ORDER_LINES Ipod Touch 1 220.95 Monster Beat 2 190.00 Apple Mouse 1 69.90 Consumer REST/SOAP Service Repository/DAO O/R Mapping SQL RDBMS 2012 © Trivadis 7 NoSQL and SOA 10.10.2012
  • 8. Relational Databases are great ... But! Problem: Semi-structured data  Relational schema doesn„t easily handle semi-structured data  Common solutions - Name/Value table - Poor performance - Lack of constraint - Serialize as Blob - Fewer joins, but no query capabilities Problem: Scaling  Scaling writes difficult/expensive/impossible => BigData  Vertical scaling is limited and is expensive  Horizontal scaling is limited and is expensive 2012 © Trivadis 8 NoSQL and SOA 10.10.2012
  • 9. Solution: NoSQL ? No standard definition of what NoSQL means • Not Only SQL Term began in a workshop organized in 2009 but some common characteristics of NoSQL databases • They don„t use the relational data model and thus don„t use SQL • Tend to be designed to run on cluster RDBMS NoSQL • Tend to be Open Source Presentation Tier User Interface User Interface • Schema-Less - Don„t have a fixed Key Value Stores schema, allowing to store any Services Caching Search Middle Tier Object-Relational Relational-Object Lucene Transactions Batch data in any record MapReduce • Different APIs Search Blobs Database Tier Transactions Batch Data Caching Triggers 2012 © Trivadis 9 NoSQL and SOA 10.10.2012
  • 10. Central vs. Application Databases Central Database Application Database • Using SQL as the integration mechanism • Only accessed by a single application between applications • Only the application using the database • applications store data in common DB needs to know about the structure • Improves communication, all applications • Easier to maintain and evolve the schema operate on consistent set of data • More freedom to choose the database • Structure ends up to be more complex • Applicable to SOA (i.e. Data Service/Entity • Changes need to be coordinated with all Service) with good Service Autonomy other applications using the database • Ready for the cloud • Side-effects (i.e. adding database index) Application 1 Application 2 Application 3 Application 1 Application 2 Application 3 DB DB DB DB 2012 © Trivadis 10 NoSQL and SOA 10.10.2012
  • 11. Relational vs. Aggregate Data Models  The relational model takes the  Aggregate is a term that comes information and divides it into from Domain-Driven Design tuples (rows) (Evans)  A tuple is a limited data structure  An aggregate is a collection of  no nesting of tuples related objects, that should be  no list of values treated as a unit  Unit for data manipulation and management of consistency 2012 © Trivadis 11 NoSQL and SOA 10.10.2012
  • 12. Relational vs. Aggregate Data Model Relational Instance Aggregate Instance CUSTOMER PRODUCT ID NAME ID NAME { 1 Guido 1000 IPod Touch „id“:1, 1020 Monster Beat „name“:“Guido“, BILLING_ADDRESS „billingAddress“:[{„street“:“Chaumontweg“,“city“:“Spiegel“,“postCode“:“3095“}] } ID CUSTOMER_ID ADDRESS_ID 1 1 55 { „id“:90, ADDRESS „customerId“:1, ID STREET CITY POST_CODE „orderItems“:[ { 55 Chaumontweg Spiegel 3095 „productId“:1000,“price“: 250.55, „produtName“: „iPod Touch“ }, ORDER { ID CUSTOMER_ID SHIPPING_ADDRESS_ID „productId“:1020,“price“: 199.55, „produtName“: „Monster Beat“ 90 1 55 }], „sippingAddress“:[{„street“:“Chaumontweg“,“city“:“Spiegel“,“postCode“:“3095“}] } ORDER_ITEM ID ORDER_ID PRODUCT_ID PRICE 1 90 1000 250.55 1 90 1020 199.55 2012 © Trivadis 12 NoSQL and SOA 10.10.2012
  • 13. Agenda 1. Why NoSQL and what is it? 2. NoSQL Database Types 3. Polyglot Persistence 4. NoSQL and Service-Oriented Architecture 5. Summary 2012 © Trivadis 15 NoSQL and SOA 10.10.2012
  • 14. NoSQL Database Types Key/Value Column Family Document Graph  Key/Value Stores Design Collections of Columns and Key/Value pairs Focus on the  Ordered Key-Value Stores Colum Families. Key/Value Pairs but value is connections Accesses directly interpreted by between data and  Big Table Stores (map-of-maps-of-maps) the column values. the database the fast navigation Document Stores Scalability/ +++ +++ ++ ++ Performance Graph Databases Aggregate- Yes Yes Yes No oriented Complexity + ++ ++ +++ Inspiration and Berkley DB, SAP Sybase IQ, Lotus Notes Graph Theory Relation Memcached, BigTable Distributed Hashmaps NoSQL Voldemort Hbase CouchDB Sones Products Redis Cassandra MongoDB Neo4J Riak Hypertable OrientDB InfoGrid Amazon SimpleDB RavenDB FlockDB 2012 © Trivadis 16 NoSQL and SOA 10.10.2012
  • 15. NoSQL Database Types Size Key-value stores Column Family Document Graph Relational Complexity 2012 © Trivadis 17 NoSQL and SOA 10.10.2012
  • 16. Key Value Databases  A key-value store is a simple hash table  Primarily used when all access to the database is via primary key  Simplest NoSQL data stores to use (from an API perspective)  PUT, GET, DELETE (matches REST)  Value is a blob with the data store not caring or knowing what is inside  Aggregate-Oriented Suitable Use Cases • Storing Session Information • User Profiles, Preferences • Shopping Cart Data 2012 © Trivadis 18 NoSQL and SOA 10.10.2012
  • 17. Column-Family Stores  Store data in column families as rows that have many columns associated with a row key  Column families are groups of related data, often accessed together  Aggregate-Oriented Suitable Use Cases • Event Logging • Content Management Systems • Counters Source: NoSQL Distilled • Expiring Usage 2012 © Trivadis 19 NoSQL and SOA 10.10.2012
  • 18. Document Databases  Documents are the main concept  Stores and retrieves documents, which can be XML, JSON, BSON, …  Documents are self-describing, hierarchical tree data structures which can consist of maps, collections and scalar values  Documents stored are similar to each other but do not have to be exactly the same  Aggregate-Oriented Suitable Use Cases • Event Logging • Content Management Systems • Web Analytics or Real-Time Analytics • Product Catalog 2012 © Trivadis 20 NoSQL and SOA 10.10.2012
  • 19. Document Database - MongoDB 2012 © Trivadis 21 NoSQL and SOA 10.10.2012
  • 20. Graph Databases  Allow to store entities and relationships between these entities  Entities are known as nodes, which have properties  Relations are known as edges, which also have properties  A query on the graph is also known as traversing the graph  Traversing the relationships is very fast Tag Customer Suitable Use Cases Country RATED TAG • Connected Data ADDRESS COUNTRY Product • Routing, Dispatch and Location-Based BILLING_ LINE_ITEM Services ADDRESS Address Recommendation Engines Order • DELIVERY_ ADDRESS 2012 © Trivadis 22 NoSQL and SOA 10.10.2012
  • 21. Graph Database – Neo4J Query through Cypher START MATCH WHERE RETURN ORDER BY LIMIT customer=node:Customer(email = "david@dmband.com") customer-[:ORDERED]->order-[item:LINEITEM]->product order.date > 20120101 product.name, sum(item.amount) AS product products DESC 20 2012 © Trivadis 23 NoSQL and SOA 10.10.2012
  • 22. Agenda 1. Why NoSQL and what is it? 2. NoSQL Database Types 3. Polyglot Persistence 4. NoSQL and Service-Oriented Architecture 5. Summary 2012 © Trivadis 24 NoSQL and SOA 10.10.2012
  • 23. Polyglot Persistence In 2006, Neal Ford coined the term Polyglot Programming  Applications should be written in a mix of languages to take advantage of the fact that different languages are suitable for tackling different problems Polyglot Persistence defines a a hybrid approach to persistence  Using multiple data storage technologies  Selected based on the way data is being used by individual applications  Why store binary images in relational databases, when there are better storage systems?  Can occur both over the enterprise as well as within a single application 2012 © Trivadis 25 NoSQL and SOA 10.10.2012
  • 24. „Traditional“ Persistence Model Polyglot Persistence E-commerce Application Today we use the same database for all kind of data Shopping cart data User Sessions Completed Order Product Catalog Recomendations • Business transactions, session management RDBMS data, reporting, logging information, content information, ... Polygot Persistence Model No need for same properties of availability, consistency or E-commerce Application backup requirements Polyglot Data Storage Usage Shopping cart data User Sessions Completed Order Product Catalog Recomendations allows to mix and match Relational and NoSQL data stores Key-Value RDMBS Document Graph 2012 © Trivadis 26 NoSQL and SOA 10.10.2012
  • 25. Polyglot Persistence – Challenges • Decisions • Have to decide what data storage technology to use • Today it„s easier to go with relational • New Data Access APIs • Each data store has its own mechanisms for accessing the data • Different API‟s Service-Oriented Polygot Persistence Model E-commerce Application • Solution: Wrap the data access code into services (Data/Entity Service) exposed to applications • Will enforce a contract/schema Shopping cart data User Sessions Completed Order Product Catalog Recomendations to a schemaless database Key-Value Graph RDMBS Document Shopping Cart User Session Product Catalog Recomendation Service Service Order Service Service Service 2012 © Trivadis 27 NoSQL and SOA 10.10.2012
  • 26. Polyglot Persistence – Challenges • Immaturity • NoSQL tools are still young, full of rough edges that new tools have • Not much experience, we don„t know how to use them well • No patterns and best practices exist yet • Organizational Change • How will the different data groups in an enterprise react to this new technology • Dealing with eventual consistency paradigm • Reaction of different stakeholders to the fact that data could be stale • How to enforce rules to sync data across systems 2012 © Trivadis 28 NoSQL and SOA 10.10.2012
  • 27. Agenda 1. What is NoSQL and Big Data 2. NoSQL Database Types 3. Polyglot Persistence 4. NoSQL and Service-Oriented Architecture 5. Summary 2012 © Trivadis 29 NoSQL and SOA 10.10.2012
  • 28. Data Access Architecture for Polyglot Persistence well known design patterns are still valid! some best practices we know in data access are still valid! Consumer Consumer Consumer REST/SOAP REST/SOAP Service Service REST Repository/DAO Repository/DAO O/R Mapping ??? SQL REST API RDBMS NoSQL NoSQL 2012 © Trivadis 30 NoSQL and SOA 10.10.2012
  • 29. Middle Tier Architecture for Polyglot Persistence Resource Tier Middle Tier Consumer Integration Service Application Domain Integration Domain Service Bean Web Service Exporter Application Service Bean REST Composite Application Factory Bean SOAP O/R Mapping Domain Objects NoSQL API Repository Bean Aggregate SQL API DAO Bean Data Transfer Objects (DTO) 2012 © Trivadis 31 NoSQL and SOA 10.10.2012
  • 30. Polyglot Persistence with Spring Data makes it easier to build Spring-powered applications that use new data access technologies provide improved support for relational database technologies Commons project supports Polyglot Persistence Currently support for: • JPA and JDBC (relational) Consumer • Apache Hadoop REST/SOAP • GemFire Service • REST Repository/DAO • Redis • MongoDB ??? • Neo4J • Hbase NoSQL 2012 © Trivadis 32 NoSQL and SOA 10.10.2012
  • 31. Spring Data – Mapping to Relational Database (using JPA) Annotations define the mapping: @Entity, @Id, @Column, @OneToOne, @OneToMany, @JoinCol umn, Consumer REST/SOAP Service Repository/DAO ??? NoSQL 2012 © Trivadis 33 NoSQL and SOA 10.10.2012
  • 32. Spring Data – Mapping to Relational Database Consumer REST/SOAP Service public interface CustomerRepository extends Repository<Customer, Long> { Repository/DAO Customer findByEmailAddress(EmailAddress emailAddress); } ??? @Repository NoSQL @Profile(“jpa") class JpaCustomerRepository implements CustomerRepository { @Override public Customer findByEmailAddress(EmailAddress emailAddress) { TypedQuery<Customer> query = em.createQuery( "select c from Customer c where c.emailAddress = :email“, Customer.class); query.setParameter("email", emailAddress); Customer guido= repository.findByEmailAddress(new return query.getSingleResult(); EmailAddress(“guido@hotmail.com")); } Customer anotherCust= new Customer(“Peter", “Sample"); <jpa:repositories base-package="com.oreilly.springdata.jpa" /> anotherCust.setEmailAddress(guido.getEmailAddress()); repository.save(anotherCust); <bean class="org.springframework.orm.jpa.JpaTransactionManager"> <property name="entityManagerFactory" ref="entityManagerFactory" /> </bean> <bean id="entityManagerFactory" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean"> <property name="dataSource" ref="dataSource" /> <property name="packagesToScan" value="com.oreilly.springdata.jpa" /> </bean> 2012 © Trivadis 34 NoSQL and SOA 10.10.2012
  • 33. Spring Data – Mapping to MongoDB Annotations define the mapping: @Document, @Id, @Indexed, @PersistenceConstructor, @Compound Index, @DBRef, @GeoSpatialIndex, @V alue Consumer REST/SOAP Service Repository/DAO ??? NoSQL 2012 © Trivadis 35 NoSQL and SOA 10.10.2012
  • 34. Spring Data – Generic Repositories for MongoDB Consumer REST/SOAP Service public interface CustomerRepository extends Repository<Customer, Long> { Repository/DAO Customer findByEmailAddress(EmailAddress emailAddress); } ??? @Repository NoSQL @Profile("mongodb") class MongoDbCustomerRepository implements CustomerRepository { @Override public Customer findByEmailAddress(EmailAddress emailAddress) { Query query = query(where("emailAddress").is(emailAddress)); return operations.findOne(query, Customer.class); } <mongo:db-factory id="mongoDbFactory" dbname="e-store" /> <mongo:mapping-converter id="mongoConverter" base-package="com.oreilly.springdata.mongodb"> <mongo:custom-converters base-package="com.oreilly.springdata.mongodb" /> </mongo:mapping-converter> <bean id="mongoTemplate" class="org.springframework.data.mongodb.core.MongoTemplate"> Customer guido= repository.findByEmailAddress(new <constructor-arg ref="mongoDbFactory" /> EmailAddress(“guido@hotmail.com")); <constructor-arg ref="mongoConverter" /> <property name="writeConcern" value="SAFE" /> Customer anotherCust= new Customer(“Peter", “Sample"); </bean> anotherCust.setEmailAddress(guido.getEmailAddress()); <mongo:repositories base-package="com.oreilly.springdata.mongodb" /> repository.save(anotherCust); 2012 © Trivadis 36 NoSQL and SOA 10.10.2012
  • 35. Spring Data – Mapping to Neo4J Annotations define the mapping: @NodeEntity, RelationShipEntity, @Gra phId, @RelatedTo, @RelatedToVia, @E ndNode, @Fetch, Tag Customer Country RATED TAG ADDRESS COUNTRY Product BILLING_ ADDRESS LINE_ITEM Address Order DELIVERY_ ADDRESS 2012 © Trivadis 37 NoSQL and SOA 10.10.2012
  • 36. Spring Data – Generic Repositories for MongoDB Consumer REST/SOAP Service Repository/DAO public interface CustomerRepository extends GraphRepository<Customer> { ??? Customer findByEmailAddress(EmailAddress emailAddress); } NoSQL <neo4j:config graphDatabaseService="graphDatabaseService" /> <neo4j:repositories base-package="com.oreilly.springdata.neo4j" /> <bean id="graphDatabaseService" class="org.neo4j.kernel.EmbeddedGraphDatabase" destroy-method="shutdown"> <constructor-arg value="target/graph.db" /> </bean> Customer guido= repository.findByEmailAddress(new EmailAddress(“guido@hotmail.com")); Customer anotherCust= new Customer(“Peter", “Sample"); anotherCust.setEmailAddress(guido.getEmailAddress()); repository.save(anotherCust); 2012 © Trivadis 38 NoSQL and SOA 10.10.2012
  • 37. Expose contract-first Web service Consumer REST/SOAP Service Repository/DAO Use any Java Web Service Framework which supports Contract-First approach ??? Can be SOAP or can be REST NoSQL Maps the data contract (WSDL or WADL) to the schemaless database Uses the different Repository implementations Must handle data migration issues together with the Repository 2012 © Trivadis 39 NoSQL and SOA 10.10.2012
  • 38. Using SOA for Integrating “old” with “new” world 2012 © Trivadis Five Cool Use Cases for the Spring Component of Oracle SOA Suite 1.10.2012
  • 39. NoSQL databases as Caching in a SOA Results are returned from cache rather than invoking always the external service  Product data is rather static, so ideal candidate for caching Proxy Business 1 Product DB Service Service 2 3 Result Cache Oracle Service Bus 2012 © Trivadis Effective Fault Handling in Oracle SOA Suite 11g 12.10.2012
  • 40. NoSQL databases as Caching in a SOA Using Mongo DB to cache results in a SOA 2012 © Trivadis 42 NoSQL and SOA 10.10.2012
  • 41. Schemaless – We still have to migrate the data! With RDMBS we are used to keep DDL scripts together Customer with DML scripts for each single data model change Name: Peter Sample First Name: Peter Last Name: Sample BillingAddress Billing Address • Has to be in sync with the data access code Street: Somestreet 10 Version 1.0 City: Somewhere Postal Code:55901 PostalCode: 55901 RDBMS has to be changed before the application is changed => possible application downtime Customer • This is what the schemaless approach of most NoSQL Name: Peter Sample FirstName: Peter DB tries to avoid LastName: Sample Transition Billing Address Version 1.0 => 2.0 Schemaless DBs still need careful migration, due to Street: Somestreet 10 City: Somewhere implicit schema in any data access code PostalCode: 55901 But a more “on-demand” approach is possible Customer • Code can read data in a way that it tolerant to First Name: Peter Last Name: Sample changes in the data‟s implicit schema and migrate Billing Address Version 2.0 the data on the next update Street: Somestreet 10 City: Somewhere PostalCode: 55901 • Similar to service versioning => gradual change 2012 © Trivadis 43 NoSQL and SOA 10.10.2012
  • 42. Agenda 1. What is NoSQL and Big Data 2. NoSQL Database Types 3. Polyglot Persistence 4. NoSQL and Service-Oriented Architecture 5. Summary 2012 © Trivadis 44 NoSQL and SOA 10.10.2012
  • 43. Pros & Cons of NoSQL compared to RDBMS Pros Cons • No O/R impedance mismatch • Lacks in tool and framework support • Can easily evolve schemas • Few other implementations => • Can represent semi-structured potential lock in info • No support for ad-hoc queries • Can represent graphs/networks (with performance) • Another/A new database in production to take care of 2012 © Trivadis 45 NoSQL and SOA 10.10.2012
  • 44. Summary Relational databases are here to stay but NoSQL offers new persistence model Polyglot Persistence will be the future Schemaless does not mean there is no data migration! => but a more on- demand model might be possible Encapsulate data access code to be able to switch databases Service-orientation provides the data contract to a NoSQL database => to make information reusable Don„t commit to a NoSQL until you have done a significant PoC Make sure that Operations people (DBAs) are on board early enough Non-relational is not new in an enterprise (OLTP vs. OLAP) 2012 © Trivadis 46 NoSQL and SOA 10.10.2012
  • 45. Further Information 2012 © Trivadis 47 NoSQL and SOA 10.10.2012
  • 46. VIELEN DANK. Trivadis Guido Schmutz guido.schmutz@trivadis.com info@trivadis.com www.trivadis.com BASEL BERN LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN 2012 © Trivadis 48 NoSQL and SOA 10.10.2012
  • 47. Possible Use Cases  NoSQL for parallel ETL?  NoSQL for modern BI  NoSQL for stateful Middletier (i.e. shopping cart)  NoSQL for aggregated master data (i.e. through REST for Web apps)  NoSQL for a CMS-Store, directly accessible through REST  NoSQL as a local Store for Mobile applications  NoSQL for Event Sourcing and CQRS architectures 2012 © Trivadis 49 NoSQL and SOA 10.10.2012