O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

HandlerSocket

614 visualizações

Publicada em

A NOSQL APPROACH TO MYSQL

Publicada em: Tecnologia
  • Login to see the comments

  • Seja a primeira pessoa a gostar disto

HandlerSocket

  1. 1. HANDLERSOCKET A NOSQL APPROACH TO MYSQL
  2. 2. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL ABOUT ME Lukasz Barulski • backend dev AKA IAmDoingEverythingExceptFrontend dev • been with DocPlanner since dinosaurs • kicks butts in MK X /lbarulski /in/lukaszbarulski
  3. 3. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL FETCHING DATA - THE MYSQL WAY
  4. 4. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL SAMPLE QUERIES — OR —
  5. 5. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL SO, WHAT’S WRONG WITH THAT?
  6. 6. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL
  7. 7. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL BUT…LET’S TEST THE SPEED • PHP 7.0.4 • MariaDB 10.0.22 500 000 records with random data +
  8. 8. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL SQL WAY | SPEED TEST $resultsInMs = [];
 $statement = $pdo->prepare('SELECT value FROM t WHERE id=:id LIMIT 1');
 for ($i = 1; $i <= 500000; ++$i)
 {
 $before = microtime(true); 
 $statement->execute(['id' => $i]);
 $data = $statement->fetchColumn(); 
 $after = microtime(true);
 $resultsInMs[] = ($after-$before)*1000;
 }
 $resultInMs = array_sum($resultsInMs)/count($resultsInMs);
 echo round($resultInMs, 3) . ' ms.' . PHP_EOL;
 echo round(stats_standard_deviation($resultsInMs, true), 3) . ' ms.' . PHP_EOL;

  9. 9. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL SQL WAY | TEST RESULTS Average time: 0.131 ms. Standard deviation: 0.012 ms.
  10. 10. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL SQL WAY… WITHOUT STEROIDS| SPEED TEST $resultsInMs = [];
 for ($i = 1; $i <= 500000; ++$i)
 {
 $before = microtime(true);
 
 $data = $pdo
 ->query('SELECT value FROM t WHERE id='.$i.' LIMIT 1')
 ->fetchColumn();
 
 $after = microtime(true);
 $resultsInMs[] = ($after-$before)*1000;
 }
 $resultInMs = array_sum($resultsInMs)/count($resultsInMs);
 echo round($resultInMs, 3) . ' ms.' . PHP_EOL;
 echo round(stats_standard_deviation($resultsInMs, true), 3) . ' ms.' . PHP_EOL;
  11. 11. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL SQL WAY… WITHOUT STEROIDS| TEST RESULTS Average time: 0.257 ms. Standard deviation: 0.074 ms.
  12. 12. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL SQL WAY | EXAMPLES SQL Prepared Statements AVG: 0.131 ms. SQL AVG: 0.257 ms. Wordpress 4.4.2 homepage: 21 SELECT 21 x 0.257 ms. = 5.397 ms. Where that (sometimes) matters? • API • High frequency operations • Complicated (long running) reports • Microservices
  13. 13. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL SIMPLE SOLUTION?
  14. 14. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL SQL WAY | SOLUTION OR A HALF-MEASURE? • Warmup • Invalidation Problems:
  15. 15. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL HANDLERSOCKET TO THE RESCUE!
  16. 16. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL HANDLERSOCKET | THEORY •MySQL based SQL servers plugin •Text-like binary protocol - not SQL queries •TCP connection •Single thread for modifying data •Multiple threads for reading data •Coexists with standard way of using MySQL
  17. 17. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL HANDLERSOCKET WAY | SPEED TEST $indexId = $hs->getIndexId('test', 't', '', 'id,value');
 $resultsInMs = [];
 for ($i = 1; $i <= 500000; ++$i)
 {
 $before = microtime(true);
 
 $hs->select($indexId, '=', [$i]);
 $data = $hs->readResponse();
 
 $after = microtime(true);
 $resultsInMs[] = ($after-$before)*1000;
 }
 $resultInMs = array_sum($resultsInMs)/(float)count($resultsInMs);
 echo round($resultInMs, 3) . ' ms.' . PHP_EOL;
 echo round(stats_standard_deviation($resultsInMs, true), 3) . ' ms.' . PHP_EOL;
  18. 18. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL HANDLERSOCKET WAY | TEST RESULTS Average time: 0.073 ms. Standard deviation: 0.009 ms.
  19. 19. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL SQL VS. HANDLERSOCKET
  20. 20. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL SQL VS. HANDLERSOCKET | AVG(FETCHING TIME) BOXING NIGHT SQL SQL PS HS 0 0,065 0,13 0,195 0,26 0,073MS 0,131MS 0,257MS
  21. 21. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL SQL VS. HANDLERSOCKET | STDEV(FETCHING TIME) BOXING NIGHT SQL SQL PS HS 0 0,02 0,04 0,06 0,08 0,009MS 0,012MS 0,074MS
  22. 22. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL OK, BUT WHY IS IT FASTER?
  23. 23. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL SQL VS. HANDLERSOCKET | LEO, WHY? Few steps less Few bytes less • No query parsing • No query analysis • No query execution plan • ~40% smaller request size • ~40% smaller response size Less CPU usage Less network traffic Based on our examples
  24. 24. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL COMMUNICATION
  25. 25. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL COMMUNICATION | THEORY • Port 9998 • Port 9999 • Read only • Multi threaded • Write allowed • Single threaded • One-line request • One-line response • Many requests on single connection
  26. 26. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL PROTOCOL
  27. 27. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL PROTOCOL BASICS | THEORY • Text-like binary protocol • Message delimiter -> n, 0x0a • Column delimiter -> t, 0x09 • Null -> 0, 0x00 • Empty string -> tt, tn • Characters [0x00 - 0x0f] -> prefixed by 0x01 and shifted by 0x40
  28. 28. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL OPEN INDEX | THEORY P <index_id> <db> <table> <index> <columns> [<fcolumns>] <index_id> Opened index identifier, integer, opened until the client connection is closed <db> Database name <table> Table name <index> Index name or "PRIMARY" to use primary key <columns> List of columns to return in response, delimited by "," [<fcolumns>] Optional argument, list of columns to filter result on
  29. 29. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL FIND DATA | THEORY <index_id> <op> <vlen> <v1> … <vn> [LIM] [IN] [FILTER …] <index_id> Opened index identifier <op> Comparison operation to use: =, >, >=, <, <= <vlen> Number parameters to compare, equal or less than number of columns in opened index <v1> Value to compare using <op> with corresponding column from index [LIM] <limit> <offset> default values: limit=1, offset=0 [IN] @ <icol> <ivlen> <iv1> ... <ivn> [FILTER …] <ftyp> <fop> <fcol> <fval>
  30. 30. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL RESPONSE | THEORY <errorcode> <numcolumns> <r1> ... <rn> <errorcode> Request has successfully executed or not. '0' means success. Non-zero means an error <numcolumns> Number of columns of the result set <r1> … <rn> Result set, n equals <numcolumns>
  31. 31. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL RESPONSE | THEORY „If <errorcode> is non-zero, <numcolumns> is always 1 and <r1> indicates a human-readable error message, though sometimes <r1> is not provided.”
  32. 32. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL SQL TO HANDLERSOCKET | THEORY SELECT value FROM t WHERE id=1 LIMIT 1 P 1 test t PRIMARY value = 0 1 1 = 1 1 0 1 600a5841de
  33. 33. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL SUMMARY
  34. 34. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL PROS | SUMMARY • Stable • High performance • Data consistency • Replication support • Not dependent on data storage engine • Shipped with MariaDB, Percona Server • Still allows to use SQL client/server protocol
  35. 35. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL CONS | SUMMARY • Filtering only using indexes • Non-standard protocol • Very simple authentication (password per port) • Lack of support for transactions • Small community • Rarely used on production
  36. 36. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL KNOWN ISSUES | SUMMARY • Persistent connections • Too many indexes (> 1024?) • Concurrent structure changes via SQL and data modification via HandlerSocket
  37. 37. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL Q’N’A Questions?
  38. 38. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL THANK YOU! docplanner.com/career Join us!

×