SlideShare uma empresa Scribd logo
1 de 67
Baixar para ler offline
SQL, NoSQL, NewSQL?
What's a developer to do?



   Chris Richardson

   Author of POJOs in Action
   Founder of CloudFoundry.com
   chris.richardson@springsource.com
   @crichardson
Overall presentation goal


 The joy and pain of
    building Java
 applications that use
 NoSQL and NewSQL

       2
About Chris




        3
(About Chris)




        4
About Chris()




        5
About Chris




        6
About Chris




        http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/




        7
About Chris


   Developer Advocate for
     CloudFoundry.com


 Signup at CloudFoundry.com
   using promo code JFokus

                            8
Agenda
o  Why NoSQL? NewSQL?
o  Persisting entities
o  Implementing queries




     2/14/12   Copyright (c) 2011 Chris Richardson. All rights reserved.
                                                                           Slide 9
Food to Go

o  Take-out food delivery
    service
o  “Launched” in 2006
o  Used a relational
    database (naturally)




         10
Success è Growth challenges
o  Increasing traffic
o  Increasing data volume
o  Distribute across a few data centers
o  Increasing domain model complexity
Limitations of relational databases

o  Scaling
o  Distribution
o  Updating schema
o  O/R impedance mismatch
o  Handling semi-structured data




                                      12
Solution: Spend Money
                                                                          o    Buy SSD and RAM
                                                                          o    Buy Oracle
                                                                          o    Buy high-end servers
                                                                          o    …


                                                                          OR
http://upload.wikimedia.org/wikipedia/commons/e/e5/Rising_Sun_Yacht.JPG




o    Hire more DevOps
o    Use application-level sharding
o    Build your own middleware
o    …

                                  http://www.trekbikes.com/us/en/bikes/road/race_performance/madone_5_series/madone_5_2/#


                            13
Solution: Use NoSQL
 Benefits




                 Higher               Limited
                 performance          transactions
                 Higher scalability   Relaxed
                 Richer data-         consistency
                 model                Unconstrained
                 Schema-less          data




                                                      Drawbacks
            14
MongoDB
o  Document-oriented database
  n  JSON-style documents: Lists, Maps, primitives
  n  Schema-less
o  Transaction = update of a single document
o  Rich query language for dynamic queries
o  Tunable writes: speed ó reliability
o  Highly scalable and available
o  Use cases
  n  High volume writes
  n  Complex data
  n  Semi-structured data

          15
Apache Cassandra
o  Column-oriented database/Extensible row store
   n    Think Row ~= java.util.SortedMap
o  Transaction = update of a row
o  Fast writes = append to a log
o  Tunable reads/writes: consistency ó latency/
    availability
o  Extremely scalable
   n    Transparent and dynamic clustering
   n    Rack and datacenter aware data replication
o  CQL = “SQL”-like DDL and DML
o  Use cases
   n    Big data
   n    Multiple Data Center distributed database
   n    (Write intensive) Logging
   n    High-availability (writes)



               16
Other NoSQL databases
Type                                   Examples


Extensible columns/Column-             Hbase
oriented                               SimpleDB
                                       DynamoDB

Graph                                  Neo4j


Key-value                              Redis
                                       Membase

Document                               CouchDb


            http://nosql-database.org/ lists 122+ NoSQL databases

                 17
Solution: Use NewSQL
o  Relational databases with SQL and
    ACID transactions
                                      AND
o  New and improved architecture
o  Radically better scalability and
    performance

o  NewSQL vendors: ScaleDB,
    NimbusDB, …, VoltDB
      2/14/12   Copyright (c) 2011 Chris Richardson. All rights reserved.
                                                                            Slide 18
Stonebraker’s motivations

                   “…Current databases are designed for
                           1970s hardware …”

                    Stonebraker: http://www.slideshare.net/VoltDB/sql-myths-webinar




  Significant overhead in “…logging, latching,
   locking, B-tree, and buffer management
                  operations…”
  SIGMOD 08: Though the looking glass: http://dl.acm.org/citation.cfm?id=1376713




                                                                                      19
About VoltDB
o  Open-source
o  In-memory relational database
o  Durability thru replication; snapshots
    and logging
o  Transparent partitioning
o  Fast and scalable
     …VoltDB is very scalable; it should scale to 120
      partitions, 39 servers, and 1.6 million complex
     transactions per second at over 300 CPU cores…
      http://www.mysqlperformanceblog.com/2011/02/28/is-voltdb-really-as-scalable-as-they-claim/




       2/14/12                  Copyright (c) 2011 Chris Richardson. All rights reserved.
                                                                                                   Slide 20
The future is polyglot persistence




                                                                           e.g. Netflix
                                                                           •  RDBMS
                                                                           •  SimpleDB
                                                                           •  Cassandra
                                                                           •  Hadoop/Hbase




   IEEE Software Sept/October 2010 - Debasish Ghosh / Twitter @debasishg



                          21
Spring Data is here to help




                                  For

                    NoSQL databases

http://www.springsource.org/spring-data



               22
Agenda
o  Why NoSQL? NewSQL?
o  Persisting entities
o  Implementing queries




      2/14/12   Copyright (c) 2011 Chris Richardson. All rights reserved.
                                                                            Slide 23
Food to Go – Place Order use case

1.  Customer enters delivery address and
    delivery time
2.  System displays available restaurants
3.  Customer picks restaurant
4.  System displays menu
5.  Customer selects menu items
6.  Customer places order



          24
Food to Go – Domain model (partial)


class Restaurant {               class TimeRange {
  long id;                         long id;
  String name;                     int dayOfWeek;
  Set<String> serviceArea;         int openingTime;
  Set<TimeRange> openingHours;
                                   int closingTime;
  List<MenuItem> menuItems;
                                 }
}


                                 class MenuItem {
                                   String name;
                                   double price;
                                 }




               25
Database schema
ID                   Name                  …
                                                         RESTAURANT
1                    Ajanta
                                                         table
2                    Montclair Eggshop

Restaurant_id            zipcode
                                                RESTAURANT_ZIPCODE
1                        94707
                                                table
1                        94619
2                        94611
2                        94619                 RESTAURANT_TIME_RANGE
                                               table

Restaurant_id   dayOfWeek           openTime     closeTime
1               Monday              1130         1430
1               Monday              1730         2130
2               Tuesday             1130         …


                26
How to implement the repository?
      public interface AvailableRestaurantRepository {

          void add(Restaurant restaurant);
          Restaurant findDetailsById(int id);
          …
      }




             Restaurant

                                                ó       ?
   TimeRange             MenuItem

Restaurant aggregate


                    27
MongoDB: persisting restaurants is easy
                                                        Server
                                   Database: Food To Go
                              Collection: Restaurants
     {
         "_id" : ObjectId("4bddc2f49d1505567c6220a0")
         "name": "Ajanta",
         "serviceArea": ["94619", "99999"],                      BSON =
         "openingHours": [
            {
                                                                 binary
               "dayOfWeek": 1,                                   JSON
              "open": 1130,
              "close": 1430 },
            {                                                    Sequence
                "dayOfWeek": 2,
                "open": 1130,
                                                                 of bytes on
                "close": 1430                                    disk è fast
             }, …
          ]
                                                                 i/o
     }




             28
Spring Data for Mongo code
@Repository
public class AvailableRestaurantRepositoryMongoDbImpl
          implements AvailableRestaurantRepository {

 public static String AVAILABLE_RESTAURANTS_COLLECTION = "availableRestaurants";

 @Autowired
 private MongoTemplate mongoTemplate;

 @Override
 public void add(Restaurant restaurant) {
   mongoTemplate.insert(restaurant, AVAILABLE_RESTAURANTS_COLLECTION);
 }

 @Override
  public Restaurant findDetailsById(int id) {
     return mongoTemplate.findOne(new Query(where("_id").is(id)),
                                   Restaurant.class,
                                   AVAILABLE_RESTAURANTS_COLLECTION);
  }
}


                     29
Spring Configuration
@Configuration
public class MongoConfig extends AbstractDatabaseConfig {

 @Value("#{mongoDbProperties.databaseName}")
 private String mongoDbDatabase;

@Bean
public Mongo mongo() throws UnknownHostException, MongoException {
  return new Mongo(databaseHostName);
}

 @Bean
 public MongoTemplate mongoTemplate(Mongo mongo) throws Exception {
    MongoTemplate mongoTemplate = new MongoTemplate(mongo, mongoDbDatabase);
    mongoTemplate.setWriteConcern(WriteConcern.SAFE);
    mongoTemplate.setWriteResultChecking(WriteResultChecking.EXCEPTION);
    return mongoTemplate;
  }
}




                     30
Cassandra data model
           Column           Column
   Row      Name             Value
   Key                               Timestamp

                                                                    Keyspace
                                                            Column Family

     K1      N1        V1     TS1    N2   V2     TS2   N3    V3   TS3




     K2      N1        V1     TS1    N2   V2     TS2   N3    V3   TS3




Column name/value: number, string, Boolean, timestamp, and composite

                  31
Cassandra– inserting/updating data
                                                                   Column Family

        K1    N1   V1   TS1   N2   V2   TS2   N3   V3   TS3




    …



Idempotent= transaction                 CF.insert(key=K1, (N4, V4, TS4), …)


                                                                   Column Family

        K1    N1   V1   TS1   N2   V2   TS2   N3   V3   TS3   N4   V4   TS4




    …


                   32
Cassandra– retrieving data
                                                              Column Family

    K1   N1   V1   TS1   N2   V2   TS2   N3   V3   TS3   N4   V4   TS4




…


                    CF.slice(key=K1, startColumn=N2, endColumn=N4)



 K1                      N2   V2   TS2   N3   V3   TS3   N4   V4   TS4




         Cassandra has secondary indexes but they
             aren’t helpful for these use cases

              33
Option #1: Use a column per attribute

       Column Name = path/expression to access property value

                                                         Column Family: RestaurantDetails

                                                              openingHours[0].dayOfWeek     Monday
       name   Ajanta           serviceArea[0]    94619

1                                                                openingHours[0].open      1130
               type    indian        serviceArea[1]   94707

                                                                 openingHours[0].close     1430




              Egg                                             openingHours[0].dayOfWeek    Monday
       name                     serviceArea[0]   94611
              shop

2                      Break                                     openingHours[0].open      0830
               type                  serviceArea[1]   94619
                        Fast

                                                               openingHours[0].close      1430
Option #2: Use a single column
      Column value = serialized object graph, e.g. JSON


                                                  Column Family: RestaurantDetails
         2          attributes: { name: “Montclair Eggshop”, … }
  1          attributes     { name: “Ajanta”, …}




  2          attributes     { name: “Eggshop”, …}




                                                                   ✔
               35
Cassandra code
public class AvailableRestaurantRepositoryCassandraKeyImpl
          implements AvailableRestaurantRepository {

@Autowired                                                    Home grown
private final CassandraTemplate cassandraTemplate;
                                                              wrapper class
public void add(Restaurant restaurant) {
  cassandraTemplate.insertEntity(keyspace,
                                 RESTAURANT_DETAILS_CF,
                                 restaurant);
}

public Restaurant findDetailsById(int id) {
  String key = Integer.toString(id);
  return cassandraTemplate.findEntity(Restaurant.class,
         keyspace, key, RESTAURANT_DETAILS_CF);
  …
}

…                                                            http://en.wikipedia.org/wiki/Hector

                  36
Using VoltDB
o  Use the original schema
o  Standard SQL statements

                   BUT YOU MUST


o  Write stored procedures and invoke
    them using proprietary interface
o  Partition your data


      2/14/12   Copyright (c) 2011 Chris Richardson. All rights reserved.
                                                                            Slide 37
About VoltDB stored procedures
o  Key part of VoltDB
o  Replication = executing stored
    procedure on replica
o  Logging = log stored procedure
    invocation
o  Stored procedure invocation =
    transaction



      2/14/12   Copyright (c) 2011 Chris Richardson. All rights reserved.
                                                                            Slide 38
About partitioning

 Partition column


                                                     RESTAURANT table

 ID                  Name                                                       …
 1                   Ajanta
 2                   Eggshop
 …




       2/14/12      Copyright (c) 2011 Chris Richardson. All rights reserved.
                                                                                    Slide 39
Example cluster


 Partition 1a                            Partition 2a                                       Partition 3a



   ID           Name     …                  ID            Name              …                 ID           Name     …
   1            Ajanta                      2             Eggshop                             …            ..
   …                                        …                                                 …



 Partition 3b                            Partition 1b                                       Partition 2b



   ID           Name     …                ID             Name          …                     ID        Name          …
   …            ..                        1              Ajanta                              2         Eggshop
   …                                      …                                                  …



VoltDB Server 1                      VoltDB Server 2                                       VoltDB Server 3


                     2/14/12   Copyright (c) 2011 Chris Richardson. All rights reserved.
                                                                                                                Slide 40
Single partition procedure: FAST
   SELECT * FROM RESTAURANT WHERE ID = 1


 High-performance lock free code



  ID   Name     …                  ID            Name              …                ID   Name         …
  1    Ajanta                      1             Eggshop                            …    ..
  …                                …                                                …




        …                                           …                                         …



VoltDB Server 1             VoltDB Server 2                                       VoltDB Server 3

            2/14/12   Copyright (c) 2011 Chris Richardson. All rights reserved.
                                                                                                  Slide 41
Multi-partition procedure: SLOWER
   SELECT * FROM RESTAURANT WHERE NAME = ‘Ajanta’


                    Communication/Coordination overhead


  ID   Name     …                      ID            Name              …                ID   Name         …
  1    Ajanta                          1             Eggshop                            …    ..
  …                                    …                                                …




        …                                               …                                         …



VoltDB Server 1                 VoltDB Server 2                                       VoltDB Server 3


            2/14/12       Copyright (c) 2011 Chris Richardson. All rights reserved.
                                                                                                      Slide 42
Chosen partitioning scheme

<partitions>
   <partition   table="restaurant" column="id"/>
   <partition   table="service_area" column="restaurant_id"/>
   <partition   table="menu_item" column="restaurant_id"/>
   <partition   table="time_range" column="restaurant_id"/>
   <partition   table="available_time_range" column="restaurant_id"/>
</partitions>




  Performance is excellent: much
  faster than MySQL

                  43
Stored procedure – AddRestaurant
@ProcInfo( singlePartition = true, partitionInfo = "Restaurant.id: 0”)
public class AddRestaurant extends VoltProcedure {
    public final SQLStmt insertRestaurant =
              new SQLStmt("INSERT INTO Restaurant VALUES (?,?);");
    public final SQLStmt insertServiceArea =
              new SQLStmt("INSERT INTO service_area VALUES (?,?);");
    public final SQLStmt insertOpeningTimes =
              new SQLStmt("INSERT INTO time_range VALUES (?,?,?,?);");
    public final SQLStmt insertMenuItem =
              new SQLStmt("INSERT INTO menu_item VALUES (?,?,?);");
public long run(int id, String name, String[] serviceArea, long[] daysOfWeek, long[] openingTimes,
                                 long[] closingTimes, String[] names, double[] prices) {
    voltQueueSQL(insertRestaurant, id, name);
    for (String zipCode : serviceArea)
      voltQueueSQL(insertServiceArea, id, zipCode);
    for (int i = 0; i < daysOfWeek.length ; i++)
      voltQueueSQL(insertOpeningTimes, id, daysOfWeek[i], openingTimes[i], closingTimes[i]);
    for (int i = 0; i < names.length ; i++)
      voltQueueSQL(insertMenuItem, id, names[i], prices[i]);
    voltExecuteSQL(true);
     return 0;
    }
}
                            44
VoltDb repository – add()
@Repository
public class AvailableRestaurantRepositoryVoltdbImpl
           implements AvailableRestaurantRepository {

    @Autowired
    private VoltDbTemplate voltDbTemplate;

    @Override
    public void add(Restaurant restaurant) {
      invokeRestaurantProcedure("AddRestaurant", restaurant);
    }

    private void invokeRestaurantProcedure(String procedureName, Restaurant restaurant) {
     Object[] serviceArea = restaurant.getServiceArea().toArray();
     long[][] openingHours = toArray(restaurant.getOpeningHours());            Flatten
     Object[][] menuItems = toArray(restaurant.getMenuItems());
                                                                             Restaurant
     voltDbTemplate.update(procedureName, restaurant.getId(), restaurant.getName(),
                               serviceArea, openingHours[0], openingHours[1],
                               openingHours[2], menuItems[0], menuItems[1]);
}



                       45
VoltDbTemplate wrapper class
public class VoltDbTemplate {

   private Client client;         VoltDB client API

   public VoltDbTemplate(Client client) {
     this.client = client;
   }

   public void update(String procedureName, Object... params) {
     try {
       ClientResponse x =
               client.callProcedure(procedureName, params);
       …
   } catch (Exception e) {
       throw new RuntimeException(e);
     }
   }

                46
VoltDb server configuration
<?xml version="1.0"?>                      <deployment>
<project>
  <info>
                                             <cluster hostcount="1"
     <name>Food To Go</name>                      sitesperhost="5" kfactor="0" />
     ...
  </info>                                  </deployment>
  <database>
     <schemas>
         <schema path='schema.sql' />
     </schemas>
     <partitions>
           <partition table="restaurant" column="id"/>
            ...
     </partitions>
     <procedures>
        <procedure class='net.chrisrichardson.foodToGo.newsql.voltdb.procs.AddRestaurant' />
        ...
     </procedures>
  </database>
</project>


voltcompiler target/classes 
   src/main/resources/sql/voltdb-project.xml foodtogo.jar


bin/voltdb leader localhost catalog foodtogo.jar deployment deployment.xml
                    47
Agenda
o  Why NoSQL? NewSQL?
o  Persisting entities
o  Implementing queries




     2/14/12   Copyright (c) 2011 Chris Richardson. All rights reserved.
                                                                           Slide 48
Finding available restaurants
Available restaurants =
      Serve the zip code of the delivery address
AND
      Are open at the delivery time

public interface AvailableRestaurantRepository {

    List<AvailableRestaurant>
          findAvailableRestaurants(Address deliveryAddress,
                                   Date deliveryTime); …
}




                   49
Finding available restaurants on Monday,
6.15pm for 94619 zip

select r.*             Straightforward
from restaurant r      three-way join
 inner join restaurant_time_range tr
   on r.id =tr.restaurant_id
 inner join restaurant_zipcode sa
   on r.id = sa.restaurant_id
Where ’94619’ = sa.zip_code
and tr.day_of_week=’monday’
and tr.openingtime <= 1815
and 1815 <= tr.closingtime


          50
MongoDB = easy to query
{
    serviceArea:"94619",                        Find a
    openingHours: {
      $elemMatch : {                            restaurant
           "dayOfWeek" : "Monday",
           "open": {$lte: 1815},                that serves
       }
           "close": {$gte: 1815}
                                                the 94619 zip
}
    }
                                                code and is
                                                open at
DBCursor cursor = collection.find(qbeObject);
while (cursor.hasNext()) {                      6.15pm on a
   DBObject o = cursor.next();
   …                                            Monday
 }


db.availableRestaurants.ensureIndex({serviceArea: 1})

                 51
MongoTemplate-based code
@Repository
public class AvailableRestaurantRepositoryMongoDbImpl
                               implements AvailableRestaurantRepository {

@Autowired private final MongoTemplate mongoTemplate;

@Override
public List<AvailableRestaurant> findAvailableRestaurants(Address deliveryAddress,
                                                          Date deliveryTime) {
 int timeOfDay = DateTimeUtil.timeOfDay(deliveryTime);
 int dayOfWeek = DateTimeUtil.dayOfWeek(deliveryTime);

    Query query = new Query(where("serviceArea").is(deliveryAddress.getZip())
         .and("openingHours”).elemMatch(where("dayOfWeek").is(dayOfWeek)
                .and("openingTime").lte(timeOfDay)
                .and("closingTime").gte(timeOfDay)));

    return mongoTemplate.find(AVAILABLE_RESTAURANTS_COLLECTION, query,
                            AvailableRestaurant.class);
}

              mongoTemplate.ensureIndex(“availableRestaurants”,
                 new Index().on("serviceArea", Order.ASCENDING));
                      52
BUT how to do this with Cassandra??!
o  How can Cassandra support a query that has




                           ?
    n  A 3-way join
    n  Multiple =
    n  > and <



è We need to implement an index

              Queries instead of data
              model drives NoSQL
              database design

         53
Simplification #1: Denormalization
Restaurant_id   Day_of_week   Open_time   Close_time   Zip_code

1               Monday        1130        1430         94707
1               Monday        1130        1430         94619
1               Monday        1730        2130         94707
1               Monday        1730        2130         94619
2               Monday        0700        1430         94619
…



       SELECT restaurant_id
        FROM time_range_zip_code
        WHERE day_of_week = ‘Monday’              Simpler query:
          AND zip_code = 94619                    §  No joins
                                                  §  Two = and two <
          AND 1815 < close_time
          AND open_time < 1815

                   54
Simplification #2: Application filtering


SELECT restaurant_id, open_time
 FROM time_range_zip_code
 WHERE day_of_week = ‘Monday’     Even simpler query
   AND zip_code = 94619           •  No joins
   AND 1815 < close_time          •  Two = and one <
   AND open_time < 1815




           55
Simplification #3: Eliminate multiple =’s with
concatenation

  Restaurant_id    Zip_dow        Open_time   Close_time

  1                94707:Monday   1130        1430
  1                94619:Monday   1130        1430
  1                94707:Monday   1730        2130
  1                94619:Monday   1730        2130
  2                94619:Monday   0700        1430
  …


 SELECT restaurant_id, open_time
  FROM time_range_zip_code
  WHERE zip_code_day_of_week = ‘94619:Monday’
    AND 1815 < close_time
                                                           key
                                  range

                  56
Column family with composite column
   names as an index
      Restaurant_id     Zip_dow              Open_time            Close_time

      1                 94707:Monday         1130                 1430
      1                 94619:Monday         1130                 1430
      1                 94707:Monday         1730                 2130
      1                 94619:Monday         1730                 2130
      2                 94619:Monday         0700                 1430
      …




                                                     Column Family: AvailableRestaurants



                                JSON FOR                                     JSON FOR
                (1430,0700,2)                                (2130,1730,1)
94619:Monday                       EGG                                         AJANTA

                                                      JSON FOR
                                     (1430,1130,1)
                                                        AJANTA
Querying with a slice
                                                         Column Family: AvailableRestaurants


                                    JSON FOR                                         JSON FOR
                    (1430,0700,2)                                (2130,1730,1)
                                       EGG                                             AJANTA
94619:Monday

                                                          JSON FOR
                                         (1430,1130,1)
                                                            AJANTA




 slice(key= 94619:Monday, sliceStart = (1815, *, *), sliceEnd = (2359, *, *))



                                                                                     JSON FOR
                                                                     (2130,1730,1)
94619:Monday                                                                           AJANTA




                                       18:15 is after 17:30 è {Ajanta}
                     58
Needs a few pages of code
         private void insertAvailability(Restaurant restaurant) {
                for (String zipCode : (Set<String>) restaurant.getServiceArea()) {
@Override         for (TimeRange tr : (Set<TimeRange>) restaurant.getOpeningHours()) {
 public List<AvailableRestaurant> findAvailableRestaurants(Address deliveryAddress, Date deliveryTime) {
                    String dayOfWeek = format2(tr.getDayOfWeek());
  int dayOfWeek = DateTimeUtil.dayOfWeek(deliveryTime);
                    String openingTime = format4(tr.getOpeningTime());
  int timeOfDay = DateTimeUtil.timeOfDay(deliveryTime);
                    String closingTime = format4(tr.getClosingTime());
  String zipCode = deliveryAddress.getZip();
  String key = formatKey(zipCode, format2(dayOfWeek));
                    String restaurantId = format8(restaurant.getId());
     HSlicePredicate<Composite> predicate = new HSlicePredicate<Composite>(new CompositeSerializer());
                        String key = formatKey(zipCode, dayOfWeek);
     Composite start = new Composite();
                        String columnValue = toJson(restaurant);
     Composite finish = new Composite();
     start.addComponent(0, format4(timeOfDay), ComponentEquality.GREATER_THAN_EQUAL);
     finish.addComponent(0, format4(2359), ComponentEquality.GREATER_THAN_EQUAL);
                   Composite columnName = new Composite();
     predicate.setRange(start, finish, false, 100);
                   columnName.add(0, closingTime);
     final List<AvailableRestaurantIndexEntry> closingAfter = new ArrayList<AvailableRestaurantIndexEntry>();
                   columnName.add(1, openingTime);
                   columnName.add(2, restaurantId);
     ColumnFamilyRowMapper<String, Composite, Object> mapper = new ColumnFamilyRowMapper<String, Composite, Object>() {

      @Override
                        ColumnFamilyUpdater<String, Composite> updater
      public Object mapRow(ColumnFamilyResult<String, Composite> results) {
                                      = compositeCloseTemplate.createUpdater(key);
        for (Composite columnName : results.getColumnNames()) {
          String openTime = columnName.get(1, new StringSerializer());
                        updater.setString(columnName, columnValue);
          String restaurantId = columnName.get(2, new StringSerializer());
          closingAfter.add(new AvailableRestaurantIndexEntry(openTime, restaurantId, results.getString(columnName)));
        }
        return null;
      }
     };
                        compositeCloseTemplate.update(updater);
                    }
     compositeCloseTemplate.queryColumns(key, predicate, mapper);
                }
     List<AvailableRestaurant> result = new LinkedList<AvailableRestaurant>();
            }
     for (AvailableRestaurantIndexEntry trIdAndAvailableRestaurant : closingAfter) {
       if (trIdAndAvailableRestaurant.isOpenBefore(timeOfDay))
         result.add(trIdAndAvailableRestaurant.getAvailableRestaurant());
     }

     return result;
 }                                        59
What did I just do to query the data?




       60
Mongo vs. Cassandra
            DC1                                                            DC2

               Shard A Master                                                        Shard B Master
MongoDB                                                Remote




                   DC1 Client                                                          DC2 Client



            DC1                                                            DC2
                                                          Async
                   Cassandra                                 Or
                                                                                       Cassandra
Cassandra
                                                           Sync


                   DC1 Client                                                          DC2 Client




             2/14/12     Copyright (c) 2011 Chris Richardson. All rights reserved.
                                                                                                      Slide 61
VoltDB - attempt #1
@ProcInfo( singlePartition = false)
public class FindAvailableRestaurants extends VoltProcedure { ... }

ERROR 10:12:03,251 [main] COMPILER: Failed to plan for statement
type(findAvailableRestaurants_with_join) select r.* from restaurant
r,time_range tr, service_area sa Where ? = sa.zip_code and r.id
=tr.restaurant_id and r.id = sa.restaurant_id and tr.day_of_week=?
and tr.open_time <= ? and ? <= tr.close_time Error: "Unable to plan
for statement. Likely statement is joining two partitioned tables in a
multi-partition statement. This is not supported at this time."
ERROR 10:12:03,251 [main] COMPILER: Catalog compilation failed.




                       Bummer!
             2/14/12     Copyright (c) 2011 Chris Richardson. All rights reserved.
                                                                                     Slide 62
VoltDB - attempt #2
@ProcInfo( singlePartition = true, partitionInfo = "Restaurant.id: 0”)
public class AddRestaurant extends VoltProcedure {

 public final SQLStmt insertAvailable=
             new SQLStmt("INSERT INTO available_time_range VALUES (?,?,?, ?, ?, ?);");

  public long run(....) {
               ...
    for (int i = 0; i < daysOfWeek.length ; i++) {
      voltQueueSQL(insertOpeningTimes, id, daysOfWeek[i], openingTimes[i], closingTimes[i]);
      for (String zipCode : serviceArea) {
       voltQueueSQL(insertAvailable, id, daysOfWeek[i], openingTimes[i],
                          closingTimes[i], zipCode, name);
      }
    }
               ...                    public final SQLStmt findAvailableRestaurants_denorm = new SQLStmt(
    voltExecuteSQL(true);                 "select restaurant_id, name from available_time_range tr " +
    return 0;                             "where ? = tr.zip_code " +
  }                                       "and tr.day_of_week=? " +
}                                         "and tr.open_time <= ? " +
                                          " and ? <= tr.close_time ");


                   Works but queries are only slightly
                          faster than MySQL!
                    2/14/12            Copyright (c) 2011 Chris Richardson. All rights reserved.
                                                                                                   Slide 63
VoltDB - attempt #3
<partitions>
   ...
   <partition table="available_time_range" column="zip_code"/>
</partitions>

@ProcInfo( singlePartition = false, ...)
public class AddRestaurant extends VoltProcedure { ... }

@ProcInfo( singlePartition = true,
             partitionInfo = "available_time_range.zip_code: 0")
public class FindAvailableRestaurants extends VoltProcedure { ... }


Queries are really fast but inserts are not L

Partitioning scheme – optimal for some use
cases but not others
            64
Summary…
o  Relational databases are great BUT there
    are limitations
o  Each NoSQL database solves some
    problems BUT
  n    Limited transactions: NoSQL = NoACID
  n    One day needing ACID è major rewrite
  n    Query-driven, denormalized database design
  n    …
o  NewSQL databases such as VoltDB provides
    SQL, ACID transactions and incredible
    performance BUT
  n  Not all operations are fast
  n  Non-JDBC API


            65
… Summary
o  Very carefully pick the NewSQL/
    NoSQL DB for your application
o  Consider a polyglot persistence
    architecture
o  Encapsulate your data access code so
    you can switch
o  Startups = avoid NewSQL/NoSQL for
    shorter time to market?


      2/14/12   Copyright (c) 2011 Chris Richardson. All rights reserved.
                                                                            Slide 66
Thank you!




 Signup at CloudFoundry.com
   using promo code JFokus
           My contact info:
           chris.richardson@springsource.com
           @crichardson


      67

Mais conteúdo relacionado

Mais procurados

Codemotion 2015 Infinispan Tech lab
Codemotion 2015 Infinispan Tech labCodemotion 2015 Infinispan Tech lab
Codemotion 2015 Infinispan Tech labUgo Landini
 
mParticle's Journey to Scylla from Cassandra
mParticle's Journey to Scylla from CassandramParticle's Journey to Scylla from Cassandra
mParticle's Journey to Scylla from CassandraScyllaDB
 
Scylla Summit 2016: Compose on Containing the Database
Scylla Summit 2016: Compose on Containing the DatabaseScylla Summit 2016: Compose on Containing the Database
Scylla Summit 2016: Compose on Containing the DatabaseScyllaDB
 
NoSQL and NewSQL: Tradeoffs between Scalable Performance & Consistency
NoSQL and NewSQL: Tradeoffs between Scalable Performance & ConsistencyNoSQL and NewSQL: Tradeoffs between Scalable Performance & Consistency
NoSQL and NewSQL: Tradeoffs between Scalable Performance & ConsistencyScyllaDB
 
Mesosphere and Contentteam: A New Way to Run Cassandra
Mesosphere and Contentteam: A New Way to Run CassandraMesosphere and Contentteam: A New Way to Run Cassandra
Mesosphere and Contentteam: A New Way to Run CassandraDataStax Academy
 
Comparing Apache Cassandra 4.0, 3.0, and ScyllaDB
Comparing Apache Cassandra 4.0, 3.0, and ScyllaDBComparing Apache Cassandra 4.0, 3.0, and ScyllaDB
Comparing Apache Cassandra 4.0, 3.0, and ScyllaDBScyllaDB
 
Voldemort on Solid State Drives
Voldemort on Solid State DrivesVoldemort on Solid State Drives
Voldemort on Solid State DrivesVinoth Chandar
 
M|18 Running MariaDB TX on Containers
M|18 Running MariaDB TX on ContainersM|18 Running MariaDB TX on Containers
M|18 Running MariaDB TX on ContainersMariaDB plc
 
Scylla Summit 2016: Why Kenshoo is about to displace Cassandra with Scylla
Scylla Summit 2016: Why Kenshoo is about to displace Cassandra with ScyllaScylla Summit 2016: Why Kenshoo is about to displace Cassandra with Scylla
Scylla Summit 2016: Why Kenshoo is about to displace Cassandra with ScyllaScyllaDB
 
Scylla Summit 2016: Outbrain Case Study - Lowering Latency While Doing 20X IO...
Scylla Summit 2016: Outbrain Case Study - Lowering Latency While Doing 20X IO...Scylla Summit 2016: Outbrain Case Study - Lowering Latency While Doing 20X IO...
Scylla Summit 2016: Outbrain Case Study - Lowering Latency While Doing 20X IO...ScyllaDB
 
Cisco: Cassandra adoption on Cisco UCS & OpenStack
Cisco: Cassandra adoption on Cisco UCS & OpenStackCisco: Cassandra adoption on Cisco UCS & OpenStack
Cisco: Cassandra adoption on Cisco UCS & OpenStackDataStax Academy
 
Workshop - How to benchmark your database
Workshop - How to benchmark your databaseWorkshop - How to benchmark your database
Workshop - How to benchmark your databaseScyllaDB
 
Postgres-XC as a Key Value Store Compared To MongoDB
Postgres-XC as a Key Value Store Compared To MongoDBPostgres-XC as a Key Value Store Compared To MongoDB
Postgres-XC as a Key Value Store Compared To MongoDBMason Sharp
 
Under the Hood of a Shard-per-Core Database Architecture
Under the Hood of a Shard-per-Core Database ArchitectureUnder the Hood of a Shard-per-Core Database Architecture
Under the Hood of a Shard-per-Core Database ArchitectureScyllaDB
 
Introducing Project Alternator - Scylla’s Open-Source DynamoDB-compatible API
Introducing Project Alternator - Scylla’s Open-Source DynamoDB-compatible APIIntroducing Project Alternator - Scylla’s Open-Source DynamoDB-compatible API
Introducing Project Alternator - Scylla’s Open-Source DynamoDB-compatible APIScyllaDB
 
Ramp-Tutorial for MYSQL Cluster - Scaling with Continuous Availability
Ramp-Tutorial for MYSQL Cluster - Scaling with Continuous AvailabilityRamp-Tutorial for MYSQL Cluster - Scaling with Continuous Availability
Ramp-Tutorial for MYSQL Cluster - Scaling with Continuous AvailabilityPythian
 
Run Cloud Native MySQL NDB Cluster in Kubernetes
Run Cloud Native MySQL NDB Cluster in KubernetesRun Cloud Native MySQL NDB Cluster in Kubernetes
Run Cloud Native MySQL NDB Cluster in KubernetesBernd Ocklin
 
Introduction to Postrges-XC
Introduction to Postrges-XCIntroduction to Postrges-XC
Introduction to Postrges-XCAshutosh Bapat
 
Scylla on Kubernetes: Introducing the Scylla Operator
Scylla on Kubernetes: Introducing the Scylla OperatorScylla on Kubernetes: Introducing the Scylla Operator
Scylla on Kubernetes: Introducing the Scylla OperatorScyllaDB
 
An Overview of Apache Cassandra
An Overview of Apache CassandraAn Overview of Apache Cassandra
An Overview of Apache CassandraDataStax
 

Mais procurados (20)

Codemotion 2015 Infinispan Tech lab
Codemotion 2015 Infinispan Tech labCodemotion 2015 Infinispan Tech lab
Codemotion 2015 Infinispan Tech lab
 
mParticle's Journey to Scylla from Cassandra
mParticle's Journey to Scylla from CassandramParticle's Journey to Scylla from Cassandra
mParticle's Journey to Scylla from Cassandra
 
Scylla Summit 2016: Compose on Containing the Database
Scylla Summit 2016: Compose on Containing the DatabaseScylla Summit 2016: Compose on Containing the Database
Scylla Summit 2016: Compose on Containing the Database
 
NoSQL and NewSQL: Tradeoffs between Scalable Performance & Consistency
NoSQL and NewSQL: Tradeoffs between Scalable Performance & ConsistencyNoSQL and NewSQL: Tradeoffs between Scalable Performance & Consistency
NoSQL and NewSQL: Tradeoffs between Scalable Performance & Consistency
 
Mesosphere and Contentteam: A New Way to Run Cassandra
Mesosphere and Contentteam: A New Way to Run CassandraMesosphere and Contentteam: A New Way to Run Cassandra
Mesosphere and Contentteam: A New Way to Run Cassandra
 
Comparing Apache Cassandra 4.0, 3.0, and ScyllaDB
Comparing Apache Cassandra 4.0, 3.0, and ScyllaDBComparing Apache Cassandra 4.0, 3.0, and ScyllaDB
Comparing Apache Cassandra 4.0, 3.0, and ScyllaDB
 
Voldemort on Solid State Drives
Voldemort on Solid State DrivesVoldemort on Solid State Drives
Voldemort on Solid State Drives
 
M|18 Running MariaDB TX on Containers
M|18 Running MariaDB TX on ContainersM|18 Running MariaDB TX on Containers
M|18 Running MariaDB TX on Containers
 
Scylla Summit 2016: Why Kenshoo is about to displace Cassandra with Scylla
Scylla Summit 2016: Why Kenshoo is about to displace Cassandra with ScyllaScylla Summit 2016: Why Kenshoo is about to displace Cassandra with Scylla
Scylla Summit 2016: Why Kenshoo is about to displace Cassandra with Scylla
 
Scylla Summit 2016: Outbrain Case Study - Lowering Latency While Doing 20X IO...
Scylla Summit 2016: Outbrain Case Study - Lowering Latency While Doing 20X IO...Scylla Summit 2016: Outbrain Case Study - Lowering Latency While Doing 20X IO...
Scylla Summit 2016: Outbrain Case Study - Lowering Latency While Doing 20X IO...
 
Cisco: Cassandra adoption on Cisco UCS & OpenStack
Cisco: Cassandra adoption on Cisco UCS & OpenStackCisco: Cassandra adoption on Cisco UCS & OpenStack
Cisco: Cassandra adoption on Cisco UCS & OpenStack
 
Workshop - How to benchmark your database
Workshop - How to benchmark your databaseWorkshop - How to benchmark your database
Workshop - How to benchmark your database
 
Postgres-XC as a Key Value Store Compared To MongoDB
Postgres-XC as a Key Value Store Compared To MongoDBPostgres-XC as a Key Value Store Compared To MongoDB
Postgres-XC as a Key Value Store Compared To MongoDB
 
Under the Hood of a Shard-per-Core Database Architecture
Under the Hood of a Shard-per-Core Database ArchitectureUnder the Hood of a Shard-per-Core Database Architecture
Under the Hood of a Shard-per-Core Database Architecture
 
Introducing Project Alternator - Scylla’s Open-Source DynamoDB-compatible API
Introducing Project Alternator - Scylla’s Open-Source DynamoDB-compatible APIIntroducing Project Alternator - Scylla’s Open-Source DynamoDB-compatible API
Introducing Project Alternator - Scylla’s Open-Source DynamoDB-compatible API
 
Ramp-Tutorial for MYSQL Cluster - Scaling with Continuous Availability
Ramp-Tutorial for MYSQL Cluster - Scaling with Continuous AvailabilityRamp-Tutorial for MYSQL Cluster - Scaling with Continuous Availability
Ramp-Tutorial for MYSQL Cluster - Scaling with Continuous Availability
 
Run Cloud Native MySQL NDB Cluster in Kubernetes
Run Cloud Native MySQL NDB Cluster in KubernetesRun Cloud Native MySQL NDB Cluster in Kubernetes
Run Cloud Native MySQL NDB Cluster in Kubernetes
 
Introduction to Postrges-XC
Introduction to Postrges-XCIntroduction to Postrges-XC
Introduction to Postrges-XC
 
Scylla on Kubernetes: Introducing the Scylla Operator
Scylla on Kubernetes: Introducing the Scylla OperatorScylla on Kubernetes: Introducing the Scylla Operator
Scylla on Kubernetes: Introducing the Scylla Operator
 
An Overview of Apache Cassandra
An Overview of Apache CassandraAn Overview of Apache Cassandra
An Overview of Apache Cassandra
 

Semelhante a SQL, NoSQL, NewSQL? What's a developer to do?

SQL? NoSQL? NewSQL?!? What’s a Java developer to do? - JDC2012 Cairo, Egypt
SQL? NoSQL? NewSQL?!? What’s a Java developer to do? - JDC2012 Cairo, EgyptSQL? NoSQL? NewSQL?!? What’s a Java developer to do? - JDC2012 Cairo, Egypt
SQL? NoSQL? NewSQL?!? What’s a Java developer to do? - JDC2012 Cairo, EgyptChris Richardson
 
Manuel Hurtado. Couchbase paradigma4oct
Manuel Hurtado. Couchbase paradigma4octManuel Hurtado. Couchbase paradigma4oct
Manuel Hurtado. Couchbase paradigma4octParadigma Digital
 
If NoSQL is your answer, you are probably asking the wrong question.
If NoSQL is your answer, you are probably asking the wrong question.If NoSQL is your answer, you are probably asking the wrong question.
If NoSQL is your answer, you are probably asking the wrong question.Lukas Smith
 
Developing polyglot persistence applications (SpringOne India 2012)
Developing polyglot persistence applications (SpringOne India 2012)Developing polyglot persistence applications (SpringOne India 2012)
Developing polyglot persistence applications (SpringOne India 2012)Chris Richardson
 
Polyglot persistence for Java developers: time to move out of the relational ...
Polyglot persistence for Java developers: time to move out of the relational ...Polyglot persistence for Java developers: time to move out of the relational ...
Polyglot persistence for Java developers: time to move out of the relational ...Chris Richardson
 
Paradigmas de procesamiento en Big Data: estado actual, tendencias y oportu...
Paradigmas de procesamiento en  Big Data: estado actual,  tendencias y oportu...Paradigmas de procesamiento en  Big Data: estado actual,  tendencias y oportu...
Paradigmas de procesamiento en Big Data: estado actual, tendencias y oportu...Facultad de Informática UCM
 
Minnebar 2013 - Scaling with Cassandra
Minnebar 2013 - Scaling with CassandraMinnebar 2013 - Scaling with Cassandra
Minnebar 2013 - Scaling with CassandraJeff Bollinger
 
R & CDK: A Sturdy Platform in the Oceans of Chemical Data}
R & CDK: A Sturdy Platform in the Oceans of Chemical Data}R & CDK: A Sturdy Platform in the Oceans of Chemical Data}
R & CDK: A Sturdy Platform in the Oceans of Chemical Data}Rajarshi Guha
 
Introduction to NoSQL
Introduction to NoSQLIntroduction to NoSQL
Introduction to NoSQLbalwinders
 
Developing polyglot persistence applications #javaone 2012
Developing polyglot persistence applications  #javaone 2012Developing polyglot persistence applications  #javaone 2012
Developing polyglot persistence applications #javaone 2012Chris Richardson
 
001 hbase introduction
001 hbase introduction001 hbase introduction
001 hbase introductionScott Miao
 
DAT322_The Nanoservices Architecture That Powers BBC Online
DAT322_The Nanoservices Architecture That Powers BBC OnlineDAT322_The Nanoservices Architecture That Powers BBC Online
DAT322_The Nanoservices Architecture That Powers BBC OnlineAmazon Web Services
 
CodeFutures - Scaling Your Database in the Cloud
CodeFutures - Scaling Your Database in the CloudCodeFutures - Scaling Your Database in the Cloud
CodeFutures - Scaling Your Database in the CloudRightScale
 
Using Spring with NoSQL databases (SpringOne China 2012)
Using Spring with NoSQL databases (SpringOne China 2012)Using Spring with NoSQL databases (SpringOne China 2012)
Using Spring with NoSQL databases (SpringOne China 2012)Chris Richardson
 
NoSQL Database
NoSQL DatabaseNoSQL Database
NoSQL DatabaseSteve Min
 
Ceph Day London 2014 - The current state of CephFS development
Ceph Day London 2014 - The current state of CephFS development Ceph Day London 2014 - The current state of CephFS development
Ceph Day London 2014 - The current state of CephFS development Ceph Community
 
Navigating NoSQL in cloudy skies
Navigating NoSQL in cloudy skiesNavigating NoSQL in cloudy skies
Navigating NoSQL in cloudy skiesshnkr_rmchndrn
 
No sqlpresentation
No sqlpresentationNo sqlpresentation
No sqlpresentationSalma Gouia
 

Semelhante a SQL, NoSQL, NewSQL? What's a developer to do? (20)

SQL? NoSQL? NewSQL?!? What’s a Java developer to do? - JDC2012 Cairo, Egypt
SQL? NoSQL? NewSQL?!? What’s a Java developer to do? - JDC2012 Cairo, EgyptSQL? NoSQL? NewSQL?!? What’s a Java developer to do? - JDC2012 Cairo, Egypt
SQL? NoSQL? NewSQL?!? What’s a Java developer to do? - JDC2012 Cairo, Egypt
 
Manuel Hurtado. Couchbase paradigma4oct
Manuel Hurtado. Couchbase paradigma4octManuel Hurtado. Couchbase paradigma4oct
Manuel Hurtado. Couchbase paradigma4oct
 
If NoSQL is your answer, you are probably asking the wrong question.
If NoSQL is your answer, you are probably asking the wrong question.If NoSQL is your answer, you are probably asking the wrong question.
If NoSQL is your answer, you are probably asking the wrong question.
 
Developing polyglot persistence applications (SpringOne India 2012)
Developing polyglot persistence applications (SpringOne India 2012)Developing polyglot persistence applications (SpringOne India 2012)
Developing polyglot persistence applications (SpringOne India 2012)
 
Polyglot persistence for Java developers: time to move out of the relational ...
Polyglot persistence for Java developers: time to move out of the relational ...Polyglot persistence for Java developers: time to move out of the relational ...
Polyglot persistence for Java developers: time to move out of the relational ...
 
Paradigmas de procesamiento en Big Data: estado actual, tendencias y oportu...
Paradigmas de procesamiento en  Big Data: estado actual,  tendencias y oportu...Paradigmas de procesamiento en  Big Data: estado actual,  tendencias y oportu...
Paradigmas de procesamiento en Big Data: estado actual, tendencias y oportu...
 
Minnebar 2013 - Scaling with Cassandra
Minnebar 2013 - Scaling with CassandraMinnebar 2013 - Scaling with Cassandra
Minnebar 2013 - Scaling with Cassandra
 
R & CDK: A Sturdy Platform in the Oceans of Chemical Data}
R & CDK: A Sturdy Platform in the Oceans of Chemical Data}R & CDK: A Sturdy Platform in the Oceans of Chemical Data}
R & CDK: A Sturdy Platform in the Oceans of Chemical Data}
 
Introduction to NoSQL
Introduction to NoSQLIntroduction to NoSQL
Introduction to NoSQL
 
Developing polyglot persistence applications #javaone 2012
Developing polyglot persistence applications  #javaone 2012Developing polyglot persistence applications  #javaone 2012
Developing polyglot persistence applications #javaone 2012
 
001 hbase introduction
001 hbase introduction001 hbase introduction
001 hbase introduction
 
Wmware NoSQL
Wmware NoSQLWmware NoSQL
Wmware NoSQL
 
DAT322_The Nanoservices Architecture That Powers BBC Online
DAT322_The Nanoservices Architecture That Powers BBC OnlineDAT322_The Nanoservices Architecture That Powers BBC Online
DAT322_The Nanoservices Architecture That Powers BBC Online
 
CodeFutures - Scaling Your Database in the Cloud
CodeFutures - Scaling Your Database in the CloudCodeFutures - Scaling Your Database in the Cloud
CodeFutures - Scaling Your Database in the Cloud
 
Using Spring with NoSQL databases (SpringOne China 2012)
Using Spring with NoSQL databases (SpringOne China 2012)Using Spring with NoSQL databases (SpringOne China 2012)
Using Spring with NoSQL databases (SpringOne China 2012)
 
NoSQL Database
NoSQL DatabaseNoSQL Database
NoSQL Database
 
NoSql Databases
NoSql DatabasesNoSql Databases
NoSql Databases
 
Ceph Day London 2014 - The current state of CephFS development
Ceph Day London 2014 - The current state of CephFS development Ceph Day London 2014 - The current state of CephFS development
Ceph Day London 2014 - The current state of CephFS development
 
Navigating NoSQL in cloudy skies
Navigating NoSQL in cloudy skiesNavigating NoSQL in cloudy skies
Navigating NoSQL in cloudy skies
 
No sqlpresentation
No sqlpresentationNo sqlpresentation
No sqlpresentation
 

Mais de Chris Richardson

The microservice architecture: what, why, when and how?
The microservice architecture: what, why, when and how?The microservice architecture: what, why, when and how?
The microservice architecture: what, why, when and how?Chris Richardson
 
More the merrier: a microservices anti-pattern
More the merrier: a microservices anti-patternMore the merrier: a microservices anti-pattern
More the merrier: a microservices anti-patternChris Richardson
 
YOW London - Considering Migrating a Monolith to Microservices? A Dark Energy...
YOW London - Considering Migrating a Monolith to Microservices? A Dark Energy...YOW London - Considering Migrating a Monolith to Microservices? A Dark Energy...
YOW London - Considering Migrating a Monolith to Microservices? A Dark Energy...Chris Richardson
 
Dark Energy, Dark Matter and the Microservices Patterns?!
Dark Energy, Dark Matter and the Microservices Patterns?!Dark Energy, Dark Matter and the Microservices Patterns?!
Dark Energy, Dark Matter and the Microservices Patterns?!Chris Richardson
 
Dark energy, dark matter and microservice architecture collaboration patterns
Dark energy, dark matter and microservice architecture collaboration patternsDark energy, dark matter and microservice architecture collaboration patterns
Dark energy, dark matter and microservice architecture collaboration patternsChris Richardson
 
Scenarios_and_Architecture_SkillsMatter_April_2022.pdf
Scenarios_and_Architecture_SkillsMatter_April_2022.pdfScenarios_and_Architecture_SkillsMatter_April_2022.pdf
Scenarios_and_Architecture_SkillsMatter_April_2022.pdfChris Richardson
 
Using patterns and pattern languages to make better architectural decisions
Using patterns and pattern languages to make better architectural decisions Using patterns and pattern languages to make better architectural decisions
Using patterns and pattern languages to make better architectural decisions Chris Richardson
 
iSAQB gathering 2021 keynote - Architectural patterns for rapid, reliable, fr...
iSAQB gathering 2021 keynote - Architectural patterns for rapid, reliable, fr...iSAQB gathering 2021 keynote - Architectural patterns for rapid, reliable, fr...
iSAQB gathering 2021 keynote - Architectural patterns for rapid, reliable, fr...Chris Richardson
 
Events to the rescue: solving distributed data problems in a microservice arc...
Events to the rescue: solving distributed data problems in a microservice arc...Events to the rescue: solving distributed data problems in a microservice arc...
Events to the rescue: solving distributed data problems in a microservice arc...Chris Richardson
 
A pattern language for microservices - June 2021
A pattern language for microservices - June 2021 A pattern language for microservices - June 2021
A pattern language for microservices - June 2021 Chris Richardson
 
QConPlus 2021: Minimizing Design Time Coupling in a Microservice Architecture
QConPlus 2021: Minimizing Design Time Coupling in a Microservice ArchitectureQConPlus 2021: Minimizing Design Time Coupling in a Microservice Architecture
QConPlus 2021: Minimizing Design Time Coupling in a Microservice ArchitectureChris Richardson
 
Mucon 2021 - Dark energy, dark matter: imperfect metaphors for designing micr...
Mucon 2021 - Dark energy, dark matter: imperfect metaphors for designing micr...Mucon 2021 - Dark energy, dark matter: imperfect metaphors for designing micr...
Mucon 2021 - Dark energy, dark matter: imperfect metaphors for designing micr...Chris Richardson
 
Designing loosely coupled services
Designing loosely coupled servicesDesigning loosely coupled services
Designing loosely coupled servicesChris Richardson
 
Microservices - an architecture that enables DevOps (T Systems DevOps day)
Microservices - an architecture that enables DevOps (T Systems DevOps day)Microservices - an architecture that enables DevOps (T Systems DevOps day)
Microservices - an architecture that enables DevOps (T Systems DevOps day)Chris Richardson
 
DDD SoCal: Decompose your monolith: Ten principles for refactoring a monolith...
DDD SoCal: Decompose your monolith: Ten principles for refactoring a monolith...DDD SoCal: Decompose your monolith: Ten principles for refactoring a monolith...
DDD SoCal: Decompose your monolith: Ten principles for refactoring a monolith...Chris Richardson
 
Decompose your monolith: Six principles for refactoring a monolith to microse...
Decompose your monolith: Six principles for refactoring a monolith to microse...Decompose your monolith: Six principles for refactoring a monolith to microse...
Decompose your monolith: Six principles for refactoring a monolith to microse...Chris Richardson
 
TDC2020 - The microservice architecture: enabling rapid, reliable, frequent a...
TDC2020 - The microservice architecture: enabling rapid, reliable, frequent a...TDC2020 - The microservice architecture: enabling rapid, reliable, frequent a...
TDC2020 - The microservice architecture: enabling rapid, reliable, frequent a...Chris Richardson
 
Overview of the Eventuate Tram Customers and Orders application
Overview of the Eventuate Tram Customers and Orders applicationOverview of the Eventuate Tram Customers and Orders application
Overview of the Eventuate Tram Customers and Orders applicationChris Richardson
 
An overview of the Eventuate Platform
An overview of the Eventuate PlatformAn overview of the Eventuate Platform
An overview of the Eventuate PlatformChris Richardson
 
#DevNexus202 Decompose your monolith
#DevNexus202 Decompose your monolith#DevNexus202 Decompose your monolith
#DevNexus202 Decompose your monolithChris Richardson
 

Mais de Chris Richardson (20)

The microservice architecture: what, why, when and how?
The microservice architecture: what, why, when and how?The microservice architecture: what, why, when and how?
The microservice architecture: what, why, when and how?
 
More the merrier: a microservices anti-pattern
More the merrier: a microservices anti-patternMore the merrier: a microservices anti-pattern
More the merrier: a microservices anti-pattern
 
YOW London - Considering Migrating a Monolith to Microservices? A Dark Energy...
YOW London - Considering Migrating a Monolith to Microservices? A Dark Energy...YOW London - Considering Migrating a Monolith to Microservices? A Dark Energy...
YOW London - Considering Migrating a Monolith to Microservices? A Dark Energy...
 
Dark Energy, Dark Matter and the Microservices Patterns?!
Dark Energy, Dark Matter and the Microservices Patterns?!Dark Energy, Dark Matter and the Microservices Patterns?!
Dark Energy, Dark Matter and the Microservices Patterns?!
 
Dark energy, dark matter and microservice architecture collaboration patterns
Dark energy, dark matter and microservice architecture collaboration patternsDark energy, dark matter and microservice architecture collaboration patterns
Dark energy, dark matter and microservice architecture collaboration patterns
 
Scenarios_and_Architecture_SkillsMatter_April_2022.pdf
Scenarios_and_Architecture_SkillsMatter_April_2022.pdfScenarios_and_Architecture_SkillsMatter_April_2022.pdf
Scenarios_and_Architecture_SkillsMatter_April_2022.pdf
 
Using patterns and pattern languages to make better architectural decisions
Using patterns and pattern languages to make better architectural decisions Using patterns and pattern languages to make better architectural decisions
Using patterns and pattern languages to make better architectural decisions
 
iSAQB gathering 2021 keynote - Architectural patterns for rapid, reliable, fr...
iSAQB gathering 2021 keynote - Architectural patterns for rapid, reliable, fr...iSAQB gathering 2021 keynote - Architectural patterns for rapid, reliable, fr...
iSAQB gathering 2021 keynote - Architectural patterns for rapid, reliable, fr...
 
Events to the rescue: solving distributed data problems in a microservice arc...
Events to the rescue: solving distributed data problems in a microservice arc...Events to the rescue: solving distributed data problems in a microservice arc...
Events to the rescue: solving distributed data problems in a microservice arc...
 
A pattern language for microservices - June 2021
A pattern language for microservices - June 2021 A pattern language for microservices - June 2021
A pattern language for microservices - June 2021
 
QConPlus 2021: Minimizing Design Time Coupling in a Microservice Architecture
QConPlus 2021: Minimizing Design Time Coupling in a Microservice ArchitectureQConPlus 2021: Minimizing Design Time Coupling in a Microservice Architecture
QConPlus 2021: Minimizing Design Time Coupling in a Microservice Architecture
 
Mucon 2021 - Dark energy, dark matter: imperfect metaphors for designing micr...
Mucon 2021 - Dark energy, dark matter: imperfect metaphors for designing micr...Mucon 2021 - Dark energy, dark matter: imperfect metaphors for designing micr...
Mucon 2021 - Dark energy, dark matter: imperfect metaphors for designing micr...
 
Designing loosely coupled services
Designing loosely coupled servicesDesigning loosely coupled services
Designing loosely coupled services
 
Microservices - an architecture that enables DevOps (T Systems DevOps day)
Microservices - an architecture that enables DevOps (T Systems DevOps day)Microservices - an architecture that enables DevOps (T Systems DevOps day)
Microservices - an architecture that enables DevOps (T Systems DevOps day)
 
DDD SoCal: Decompose your monolith: Ten principles for refactoring a monolith...
DDD SoCal: Decompose your monolith: Ten principles for refactoring a monolith...DDD SoCal: Decompose your monolith: Ten principles for refactoring a monolith...
DDD SoCal: Decompose your monolith: Ten principles for refactoring a monolith...
 
Decompose your monolith: Six principles for refactoring a monolith to microse...
Decompose your monolith: Six principles for refactoring a monolith to microse...Decompose your monolith: Six principles for refactoring a monolith to microse...
Decompose your monolith: Six principles for refactoring a monolith to microse...
 
TDC2020 - The microservice architecture: enabling rapid, reliable, frequent a...
TDC2020 - The microservice architecture: enabling rapid, reliable, frequent a...TDC2020 - The microservice architecture: enabling rapid, reliable, frequent a...
TDC2020 - The microservice architecture: enabling rapid, reliable, frequent a...
 
Overview of the Eventuate Tram Customers and Orders application
Overview of the Eventuate Tram Customers and Orders applicationOverview of the Eventuate Tram Customers and Orders application
Overview of the Eventuate Tram Customers and Orders application
 
An overview of the Eventuate Platform
An overview of the Eventuate PlatformAn overview of the Eventuate Platform
An overview of the Eventuate Platform
 
#DevNexus202 Decompose your monolith
#DevNexus202 Decompose your monolith#DevNexus202 Decompose your monolith
#DevNexus202 Decompose your monolith
 

Último

The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxLoriGlavin3
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxBkGupta21
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Mattias Andersson
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyAlfredo García Lavilla
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESmohitsingh558521
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 

Último (20)

The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptx
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easy
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 

SQL, NoSQL, NewSQL? What's a developer to do?

  • 1. SQL, NoSQL, NewSQL? What's a developer to do? Chris Richardson Author of POJOs in Action Founder of CloudFoundry.com chris.richardson@springsource.com @crichardson
  • 2. Overall presentation goal The joy and pain of building Java applications that use NoSQL and NewSQL 2
  • 7. About Chris http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/ 7
  • 8. About Chris Developer Advocate for CloudFoundry.com Signup at CloudFoundry.com using promo code JFokus 8
  • 9. Agenda o  Why NoSQL? NewSQL? o  Persisting entities o  Implementing queries 2/14/12 Copyright (c) 2011 Chris Richardson. All rights reserved. Slide 9
  • 10. Food to Go o  Take-out food delivery service o  “Launched” in 2006 o  Used a relational database (naturally) 10
  • 11. Success è Growth challenges o  Increasing traffic o  Increasing data volume o  Distribute across a few data centers o  Increasing domain model complexity
  • 12. Limitations of relational databases o  Scaling o  Distribution o  Updating schema o  O/R impedance mismatch o  Handling semi-structured data 12
  • 13. Solution: Spend Money o  Buy SSD and RAM o  Buy Oracle o  Buy high-end servers o  … OR http://upload.wikimedia.org/wikipedia/commons/e/e5/Rising_Sun_Yacht.JPG o  Hire more DevOps o  Use application-level sharding o  Build your own middleware o  … http://www.trekbikes.com/us/en/bikes/road/race_performance/madone_5_series/madone_5_2/# 13
  • 14. Solution: Use NoSQL Benefits Higher Limited performance transactions Higher scalability Relaxed Richer data- consistency model Unconstrained Schema-less data Drawbacks 14
  • 15. MongoDB o  Document-oriented database n  JSON-style documents: Lists, Maps, primitives n  Schema-less o  Transaction = update of a single document o  Rich query language for dynamic queries o  Tunable writes: speed ó reliability o  Highly scalable and available o  Use cases n  High volume writes n  Complex data n  Semi-structured data 15
  • 16. Apache Cassandra o  Column-oriented database/Extensible row store n  Think Row ~= java.util.SortedMap o  Transaction = update of a row o  Fast writes = append to a log o  Tunable reads/writes: consistency ó latency/ availability o  Extremely scalable n  Transparent and dynamic clustering n  Rack and datacenter aware data replication o  CQL = “SQL”-like DDL and DML o  Use cases n  Big data n  Multiple Data Center distributed database n  (Write intensive) Logging n  High-availability (writes) 16
  • 17. Other NoSQL databases Type Examples Extensible columns/Column- Hbase oriented SimpleDB DynamoDB Graph Neo4j Key-value Redis Membase Document CouchDb http://nosql-database.org/ lists 122+ NoSQL databases 17
  • 18. Solution: Use NewSQL o  Relational databases with SQL and ACID transactions AND o  New and improved architecture o  Radically better scalability and performance o  NewSQL vendors: ScaleDB, NimbusDB, …, VoltDB 2/14/12 Copyright (c) 2011 Chris Richardson. All rights reserved. Slide 18
  • 19. Stonebraker’s motivations “…Current databases are designed for 1970s hardware …” Stonebraker: http://www.slideshare.net/VoltDB/sql-myths-webinar Significant overhead in “…logging, latching, locking, B-tree, and buffer management operations…” SIGMOD 08: Though the looking glass: http://dl.acm.org/citation.cfm?id=1376713 19
  • 20. About VoltDB o  Open-source o  In-memory relational database o  Durability thru replication; snapshots and logging o  Transparent partitioning o  Fast and scalable …VoltDB is very scalable; it should scale to 120 partitions, 39 servers, and 1.6 million complex transactions per second at over 300 CPU cores… http://www.mysqlperformanceblog.com/2011/02/28/is-voltdb-really-as-scalable-as-they-claim/ 2/14/12 Copyright (c) 2011 Chris Richardson. All rights reserved. Slide 20
  • 21. The future is polyglot persistence e.g. Netflix •  RDBMS •  SimpleDB •  Cassandra •  Hadoop/Hbase IEEE Software Sept/October 2010 - Debasish Ghosh / Twitter @debasishg 21
  • 22. Spring Data is here to help For NoSQL databases http://www.springsource.org/spring-data 22
  • 23. Agenda o  Why NoSQL? NewSQL? o  Persisting entities o  Implementing queries 2/14/12 Copyright (c) 2011 Chris Richardson. All rights reserved. Slide 23
  • 24. Food to Go – Place Order use case 1.  Customer enters delivery address and delivery time 2.  System displays available restaurants 3.  Customer picks restaurant 4.  System displays menu 5.  Customer selects menu items 6.  Customer places order 24
  • 25. Food to Go – Domain model (partial) class Restaurant { class TimeRange { long id; long id; String name; int dayOfWeek; Set<String> serviceArea; int openingTime; Set<TimeRange> openingHours; int closingTime; List<MenuItem> menuItems; } } class MenuItem { String name; double price; } 25
  • 26. Database schema ID Name … RESTAURANT 1 Ajanta table 2 Montclair Eggshop Restaurant_id zipcode RESTAURANT_ZIPCODE 1 94707 table 1 94619 2 94611 2 94619 RESTAURANT_TIME_RANGE table Restaurant_id dayOfWeek openTime closeTime 1 Monday 1130 1430 1 Monday 1730 2130 2 Tuesday 1130 … 26
  • 27. How to implement the repository? public interface AvailableRestaurantRepository { void add(Restaurant restaurant); Restaurant findDetailsById(int id); … } Restaurant ó ? TimeRange MenuItem Restaurant aggregate 27
  • 28. MongoDB: persisting restaurants is easy Server Database: Food To Go Collection: Restaurants { "_id" : ObjectId("4bddc2f49d1505567c6220a0") "name": "Ajanta", "serviceArea": ["94619", "99999"], BSON = "openingHours": [ { binary "dayOfWeek": 1, JSON "open": 1130, "close": 1430 }, { Sequence "dayOfWeek": 2, "open": 1130, of bytes on "close": 1430 disk è fast }, … ] i/o } 28
  • 29. Spring Data for Mongo code @Repository public class AvailableRestaurantRepositoryMongoDbImpl implements AvailableRestaurantRepository { public static String AVAILABLE_RESTAURANTS_COLLECTION = "availableRestaurants"; @Autowired private MongoTemplate mongoTemplate; @Override public void add(Restaurant restaurant) { mongoTemplate.insert(restaurant, AVAILABLE_RESTAURANTS_COLLECTION); } @Override public Restaurant findDetailsById(int id) { return mongoTemplate.findOne(new Query(where("_id").is(id)), Restaurant.class, AVAILABLE_RESTAURANTS_COLLECTION); } } 29
  • 30. Spring Configuration @Configuration public class MongoConfig extends AbstractDatabaseConfig { @Value("#{mongoDbProperties.databaseName}") private String mongoDbDatabase; @Bean public Mongo mongo() throws UnknownHostException, MongoException { return new Mongo(databaseHostName); } @Bean public MongoTemplate mongoTemplate(Mongo mongo) throws Exception { MongoTemplate mongoTemplate = new MongoTemplate(mongo, mongoDbDatabase); mongoTemplate.setWriteConcern(WriteConcern.SAFE); mongoTemplate.setWriteResultChecking(WriteResultChecking.EXCEPTION); return mongoTemplate; } } 30
  • 31. Cassandra data model Column Column Row Name Value Key Timestamp Keyspace Column Family K1 N1 V1 TS1 N2 V2 TS2 N3 V3 TS3 K2 N1 V1 TS1 N2 V2 TS2 N3 V3 TS3 Column name/value: number, string, Boolean, timestamp, and composite 31
  • 32. Cassandra– inserting/updating data Column Family K1 N1 V1 TS1 N2 V2 TS2 N3 V3 TS3 … Idempotent= transaction CF.insert(key=K1, (N4, V4, TS4), …) Column Family K1 N1 V1 TS1 N2 V2 TS2 N3 V3 TS3 N4 V4 TS4 … 32
  • 33. Cassandra– retrieving data Column Family K1 N1 V1 TS1 N2 V2 TS2 N3 V3 TS3 N4 V4 TS4 … CF.slice(key=K1, startColumn=N2, endColumn=N4) K1 N2 V2 TS2 N3 V3 TS3 N4 V4 TS4 Cassandra has secondary indexes but they aren’t helpful for these use cases 33
  • 34. Option #1: Use a column per attribute Column Name = path/expression to access property value Column Family: RestaurantDetails openingHours[0].dayOfWeek Monday name Ajanta serviceArea[0] 94619 1 openingHours[0].open 1130 type indian serviceArea[1] 94707 openingHours[0].close 1430 Egg openingHours[0].dayOfWeek Monday name serviceArea[0] 94611 shop 2 Break openingHours[0].open 0830 type serviceArea[1] 94619 Fast openingHours[0].close 1430
  • 35. Option #2: Use a single column Column value = serialized object graph, e.g. JSON Column Family: RestaurantDetails 2 attributes: { name: “Montclair Eggshop”, … } 1 attributes { name: “Ajanta”, …} 2 attributes { name: “Eggshop”, …} ✔ 35
  • 36. Cassandra code public class AvailableRestaurantRepositoryCassandraKeyImpl implements AvailableRestaurantRepository { @Autowired Home grown private final CassandraTemplate cassandraTemplate; wrapper class public void add(Restaurant restaurant) { cassandraTemplate.insertEntity(keyspace, RESTAURANT_DETAILS_CF, restaurant); } public Restaurant findDetailsById(int id) { String key = Integer.toString(id); return cassandraTemplate.findEntity(Restaurant.class, keyspace, key, RESTAURANT_DETAILS_CF); … } … http://en.wikipedia.org/wiki/Hector 36
  • 37. Using VoltDB o  Use the original schema o  Standard SQL statements BUT YOU MUST o  Write stored procedures and invoke them using proprietary interface o  Partition your data 2/14/12 Copyright (c) 2011 Chris Richardson. All rights reserved. Slide 37
  • 38. About VoltDB stored procedures o  Key part of VoltDB o  Replication = executing stored procedure on replica o  Logging = log stored procedure invocation o  Stored procedure invocation = transaction 2/14/12 Copyright (c) 2011 Chris Richardson. All rights reserved. Slide 38
  • 39. About partitioning Partition column RESTAURANT table ID Name … 1 Ajanta 2 Eggshop … 2/14/12 Copyright (c) 2011 Chris Richardson. All rights reserved. Slide 39
  • 40. Example cluster Partition 1a Partition 2a Partition 3a ID Name … ID Name … ID Name … 1 Ajanta 2 Eggshop … .. … … … Partition 3b Partition 1b Partition 2b ID Name … ID Name … ID Name … … .. 1 Ajanta 2 Eggshop … … … VoltDB Server 1 VoltDB Server 2 VoltDB Server 3 2/14/12 Copyright (c) 2011 Chris Richardson. All rights reserved. Slide 40
  • 41. Single partition procedure: FAST SELECT * FROM RESTAURANT WHERE ID = 1 High-performance lock free code ID Name … ID Name … ID Name … 1 Ajanta 1 Eggshop … .. … … … … … … VoltDB Server 1 VoltDB Server 2 VoltDB Server 3 2/14/12 Copyright (c) 2011 Chris Richardson. All rights reserved. Slide 41
  • 42. Multi-partition procedure: SLOWER SELECT * FROM RESTAURANT WHERE NAME = ‘Ajanta’ Communication/Coordination overhead ID Name … ID Name … ID Name … 1 Ajanta 1 Eggshop … .. … … … … … … VoltDB Server 1 VoltDB Server 2 VoltDB Server 3 2/14/12 Copyright (c) 2011 Chris Richardson. All rights reserved. Slide 42
  • 43. Chosen partitioning scheme <partitions> <partition table="restaurant" column="id"/> <partition table="service_area" column="restaurant_id"/> <partition table="menu_item" column="restaurant_id"/> <partition table="time_range" column="restaurant_id"/> <partition table="available_time_range" column="restaurant_id"/> </partitions> Performance is excellent: much faster than MySQL 43
  • 44. Stored procedure – AddRestaurant @ProcInfo( singlePartition = true, partitionInfo = "Restaurant.id: 0”) public class AddRestaurant extends VoltProcedure { public final SQLStmt insertRestaurant = new SQLStmt("INSERT INTO Restaurant VALUES (?,?);"); public final SQLStmt insertServiceArea = new SQLStmt("INSERT INTO service_area VALUES (?,?);"); public final SQLStmt insertOpeningTimes = new SQLStmt("INSERT INTO time_range VALUES (?,?,?,?);"); public final SQLStmt insertMenuItem = new SQLStmt("INSERT INTO menu_item VALUES (?,?,?);"); public long run(int id, String name, String[] serviceArea, long[] daysOfWeek, long[] openingTimes, long[] closingTimes, String[] names, double[] prices) { voltQueueSQL(insertRestaurant, id, name); for (String zipCode : serviceArea) voltQueueSQL(insertServiceArea, id, zipCode); for (int i = 0; i < daysOfWeek.length ; i++) voltQueueSQL(insertOpeningTimes, id, daysOfWeek[i], openingTimes[i], closingTimes[i]); for (int i = 0; i < names.length ; i++) voltQueueSQL(insertMenuItem, id, names[i], prices[i]); voltExecuteSQL(true); return 0; } } 44
  • 45. VoltDb repository – add() @Repository public class AvailableRestaurantRepositoryVoltdbImpl implements AvailableRestaurantRepository { @Autowired private VoltDbTemplate voltDbTemplate; @Override public void add(Restaurant restaurant) { invokeRestaurantProcedure("AddRestaurant", restaurant); } private void invokeRestaurantProcedure(String procedureName, Restaurant restaurant) { Object[] serviceArea = restaurant.getServiceArea().toArray(); long[][] openingHours = toArray(restaurant.getOpeningHours()); Flatten Object[][] menuItems = toArray(restaurant.getMenuItems()); Restaurant voltDbTemplate.update(procedureName, restaurant.getId(), restaurant.getName(), serviceArea, openingHours[0], openingHours[1], openingHours[2], menuItems[0], menuItems[1]); } 45
  • 46. VoltDbTemplate wrapper class public class VoltDbTemplate { private Client client; VoltDB client API public VoltDbTemplate(Client client) { this.client = client; } public void update(String procedureName, Object... params) { try { ClientResponse x = client.callProcedure(procedureName, params); … } catch (Exception e) { throw new RuntimeException(e); } } 46
  • 47. VoltDb server configuration <?xml version="1.0"?> <deployment> <project> <info> <cluster hostcount="1" <name>Food To Go</name> sitesperhost="5" kfactor="0" /> ... </info> </deployment> <database> <schemas> <schema path='schema.sql' /> </schemas> <partitions> <partition table="restaurant" column="id"/> ... </partitions> <procedures> <procedure class='net.chrisrichardson.foodToGo.newsql.voltdb.procs.AddRestaurant' /> ... </procedures> </database> </project> voltcompiler target/classes src/main/resources/sql/voltdb-project.xml foodtogo.jar bin/voltdb leader localhost catalog foodtogo.jar deployment deployment.xml 47
  • 48. Agenda o  Why NoSQL? NewSQL? o  Persisting entities o  Implementing queries 2/14/12 Copyright (c) 2011 Chris Richardson. All rights reserved. Slide 48
  • 49. Finding available restaurants Available restaurants = Serve the zip code of the delivery address AND Are open at the delivery time public interface AvailableRestaurantRepository { List<AvailableRestaurant> findAvailableRestaurants(Address deliveryAddress, Date deliveryTime); … } 49
  • 50. Finding available restaurants on Monday, 6.15pm for 94619 zip select r.* Straightforward from restaurant r three-way join inner join restaurant_time_range tr on r.id =tr.restaurant_id inner join restaurant_zipcode sa on r.id = sa.restaurant_id Where ’94619’ = sa.zip_code and tr.day_of_week=’monday’ and tr.openingtime <= 1815 and 1815 <= tr.closingtime 50
  • 51. MongoDB = easy to query { serviceArea:"94619", Find a openingHours: { $elemMatch : { restaurant "dayOfWeek" : "Monday", "open": {$lte: 1815}, that serves } "close": {$gte: 1815} the 94619 zip } } code and is open at DBCursor cursor = collection.find(qbeObject); while (cursor.hasNext()) { 6.15pm on a DBObject o = cursor.next(); … Monday } db.availableRestaurants.ensureIndex({serviceArea: 1}) 51
  • 52. MongoTemplate-based code @Repository public class AvailableRestaurantRepositoryMongoDbImpl implements AvailableRestaurantRepository { @Autowired private final MongoTemplate mongoTemplate; @Override public List<AvailableRestaurant> findAvailableRestaurants(Address deliveryAddress, Date deliveryTime) { int timeOfDay = DateTimeUtil.timeOfDay(deliveryTime); int dayOfWeek = DateTimeUtil.dayOfWeek(deliveryTime); Query query = new Query(where("serviceArea").is(deliveryAddress.getZip()) .and("openingHours”).elemMatch(where("dayOfWeek").is(dayOfWeek) .and("openingTime").lte(timeOfDay) .and("closingTime").gte(timeOfDay))); return mongoTemplate.find(AVAILABLE_RESTAURANTS_COLLECTION, query, AvailableRestaurant.class); } mongoTemplate.ensureIndex(“availableRestaurants”, new Index().on("serviceArea", Order.ASCENDING)); 52
  • 53. BUT how to do this with Cassandra??! o  How can Cassandra support a query that has ? n  A 3-way join n  Multiple = n  > and < è We need to implement an index Queries instead of data model drives NoSQL database design 53
  • 54. Simplification #1: Denormalization Restaurant_id Day_of_week Open_time Close_time Zip_code 1 Monday 1130 1430 94707 1 Monday 1130 1430 94619 1 Monday 1730 2130 94707 1 Monday 1730 2130 94619 2 Monday 0700 1430 94619 … SELECT restaurant_id FROM time_range_zip_code WHERE day_of_week = ‘Monday’ Simpler query: AND zip_code = 94619 §  No joins §  Two = and two < AND 1815 < close_time AND open_time < 1815 54
  • 55. Simplification #2: Application filtering SELECT restaurant_id, open_time FROM time_range_zip_code WHERE day_of_week = ‘Monday’ Even simpler query AND zip_code = 94619 •  No joins AND 1815 < close_time •  Two = and one < AND open_time < 1815 55
  • 56. Simplification #3: Eliminate multiple =’s with concatenation Restaurant_id Zip_dow Open_time Close_time 1 94707:Monday 1130 1430 1 94619:Monday 1130 1430 1 94707:Monday 1730 2130 1 94619:Monday 1730 2130 2 94619:Monday 0700 1430 … SELECT restaurant_id, open_time FROM time_range_zip_code WHERE zip_code_day_of_week = ‘94619:Monday’ AND 1815 < close_time key range 56
  • 57. Column family with composite column names as an index Restaurant_id Zip_dow Open_time Close_time 1 94707:Monday 1130 1430 1 94619:Monday 1130 1430 1 94707:Monday 1730 2130 1 94619:Monday 1730 2130 2 94619:Monday 0700 1430 … Column Family: AvailableRestaurants JSON FOR JSON FOR (1430,0700,2) (2130,1730,1) 94619:Monday EGG AJANTA JSON FOR (1430,1130,1) AJANTA
  • 58. Querying with a slice Column Family: AvailableRestaurants JSON FOR JSON FOR (1430,0700,2) (2130,1730,1) EGG AJANTA 94619:Monday JSON FOR (1430,1130,1) AJANTA slice(key= 94619:Monday, sliceStart = (1815, *, *), sliceEnd = (2359, *, *)) JSON FOR (2130,1730,1) 94619:Monday AJANTA 18:15 is after 17:30 è {Ajanta} 58
  • 59. Needs a few pages of code private void insertAvailability(Restaurant restaurant) { for (String zipCode : (Set<String>) restaurant.getServiceArea()) { @Override for (TimeRange tr : (Set<TimeRange>) restaurant.getOpeningHours()) { public List<AvailableRestaurant> findAvailableRestaurants(Address deliveryAddress, Date deliveryTime) { String dayOfWeek = format2(tr.getDayOfWeek()); int dayOfWeek = DateTimeUtil.dayOfWeek(deliveryTime); String openingTime = format4(tr.getOpeningTime()); int timeOfDay = DateTimeUtil.timeOfDay(deliveryTime); String closingTime = format4(tr.getClosingTime()); String zipCode = deliveryAddress.getZip(); String key = formatKey(zipCode, format2(dayOfWeek)); String restaurantId = format8(restaurant.getId()); HSlicePredicate<Composite> predicate = new HSlicePredicate<Composite>(new CompositeSerializer()); String key = formatKey(zipCode, dayOfWeek); Composite start = new Composite(); String columnValue = toJson(restaurant); Composite finish = new Composite(); start.addComponent(0, format4(timeOfDay), ComponentEquality.GREATER_THAN_EQUAL); finish.addComponent(0, format4(2359), ComponentEquality.GREATER_THAN_EQUAL); Composite columnName = new Composite(); predicate.setRange(start, finish, false, 100); columnName.add(0, closingTime); final List<AvailableRestaurantIndexEntry> closingAfter = new ArrayList<AvailableRestaurantIndexEntry>(); columnName.add(1, openingTime); columnName.add(2, restaurantId); ColumnFamilyRowMapper<String, Composite, Object> mapper = new ColumnFamilyRowMapper<String, Composite, Object>() { @Override ColumnFamilyUpdater<String, Composite> updater public Object mapRow(ColumnFamilyResult<String, Composite> results) { = compositeCloseTemplate.createUpdater(key); for (Composite columnName : results.getColumnNames()) { String openTime = columnName.get(1, new StringSerializer()); updater.setString(columnName, columnValue); String restaurantId = columnName.get(2, new StringSerializer()); closingAfter.add(new AvailableRestaurantIndexEntry(openTime, restaurantId, results.getString(columnName))); } return null; } }; compositeCloseTemplate.update(updater); } compositeCloseTemplate.queryColumns(key, predicate, mapper); } List<AvailableRestaurant> result = new LinkedList<AvailableRestaurant>(); } for (AvailableRestaurantIndexEntry trIdAndAvailableRestaurant : closingAfter) { if (trIdAndAvailableRestaurant.isOpenBefore(timeOfDay)) result.add(trIdAndAvailableRestaurant.getAvailableRestaurant()); } return result; } 59
  • 60. What did I just do to query the data? 60
  • 61. Mongo vs. Cassandra DC1 DC2 Shard A Master Shard B Master MongoDB Remote DC1 Client DC2 Client DC1 DC2 Async Cassandra Or Cassandra Cassandra Sync DC1 Client DC2 Client 2/14/12 Copyright (c) 2011 Chris Richardson. All rights reserved. Slide 61
  • 62. VoltDB - attempt #1 @ProcInfo( singlePartition = false) public class FindAvailableRestaurants extends VoltProcedure { ... } ERROR 10:12:03,251 [main] COMPILER: Failed to plan for statement type(findAvailableRestaurants_with_join) select r.* from restaurant r,time_range tr, service_area sa Where ? = sa.zip_code and r.id =tr.restaurant_id and r.id = sa.restaurant_id and tr.day_of_week=? and tr.open_time <= ? and ? <= tr.close_time Error: "Unable to plan for statement. Likely statement is joining two partitioned tables in a multi-partition statement. This is not supported at this time." ERROR 10:12:03,251 [main] COMPILER: Catalog compilation failed. Bummer! 2/14/12 Copyright (c) 2011 Chris Richardson. All rights reserved. Slide 62
  • 63. VoltDB - attempt #2 @ProcInfo( singlePartition = true, partitionInfo = "Restaurant.id: 0”) public class AddRestaurant extends VoltProcedure { public final SQLStmt insertAvailable= new SQLStmt("INSERT INTO available_time_range VALUES (?,?,?, ?, ?, ?);"); public long run(....) { ... for (int i = 0; i < daysOfWeek.length ; i++) { voltQueueSQL(insertOpeningTimes, id, daysOfWeek[i], openingTimes[i], closingTimes[i]); for (String zipCode : serviceArea) { voltQueueSQL(insertAvailable, id, daysOfWeek[i], openingTimes[i], closingTimes[i], zipCode, name); } } ... public final SQLStmt findAvailableRestaurants_denorm = new SQLStmt( voltExecuteSQL(true); "select restaurant_id, name from available_time_range tr " + return 0; "where ? = tr.zip_code " + } "and tr.day_of_week=? " + } "and tr.open_time <= ? " + " and ? <= tr.close_time "); Works but queries are only slightly faster than MySQL! 2/14/12 Copyright (c) 2011 Chris Richardson. All rights reserved. Slide 63
  • 64. VoltDB - attempt #3 <partitions> ... <partition table="available_time_range" column="zip_code"/> </partitions> @ProcInfo( singlePartition = false, ...) public class AddRestaurant extends VoltProcedure { ... } @ProcInfo( singlePartition = true, partitionInfo = "available_time_range.zip_code: 0") public class FindAvailableRestaurants extends VoltProcedure { ... } Queries are really fast but inserts are not L Partitioning scheme – optimal for some use cases but not others 64
  • 65. Summary… o  Relational databases are great BUT there are limitations o  Each NoSQL database solves some problems BUT n  Limited transactions: NoSQL = NoACID n  One day needing ACID è major rewrite n  Query-driven, denormalized database design n  … o  NewSQL databases such as VoltDB provides SQL, ACID transactions and incredible performance BUT n  Not all operations are fast n  Non-JDBC API 65
  • 66. … Summary o  Very carefully pick the NewSQL/ NoSQL DB for your application o  Consider a polyglot persistence architecture o  Encapsulate your data access code so you can switch o  Startups = avoid NewSQL/NoSQL for shorter time to market? 2/14/12 Copyright (c) 2011 Chris Richardson. All rights reserved. Slide 66
  • 67. Thank you! Signup at CloudFoundry.com using promo code JFokus My contact info: chris.richardson@springsource.com @crichardson 67