2. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL
ABOUT ME
Lukasz Barulski
• backend dev AKA IAmDoingEverythingExceptFrontend dev
• been with DocPlanner since dinosaurs
• kicks butts in MK X
/lbarulski /in/lukaszbarulski
3. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL
FETCHING DATA - THE MYSQL WAY
14. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL
SQL WAY | SOLUTION OR A HALF-MEASURE?
• Warmup
• Invalidation
Problems:
15. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL
HANDLERSOCKET TO THE RESCUE!
16. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL
HANDLERSOCKET | THEORY
•MySQL based SQL servers plugin
•Text-like binary protocol - not SQL queries
•TCP connection
•Single thread for modifying data
•Multiple threads for reading data
•Coexists with standard way of using MySQL
23. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL
SQL VS. HANDLERSOCKET | LEO, WHY?
Few steps less
Few bytes less
• No query parsing
• No query analysis
• No query execution plan
• ~40% smaller request size
• ~40% smaller response size
Less CPU usage
Less network traffic
Based on our examples
25. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL
COMMUNICATION | THEORY
• Port 9998
• Port 9999
• Read only
• Multi threaded
• Write allowed
• Single threaded
• One-line request
• One-line response
• Many requests on single connection
27. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL
PROTOCOL BASICS | THEORY
• Text-like binary protocol
• Message delimiter -> n, 0x0a
• Column delimiter -> t, 0x09
• Null -> 0, 0x00
• Empty string -> tt, tn
• Characters [0x00 - 0x0f] -> prefixed by 0x01 and shifted by 0x40
28. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL
OPEN INDEX | THEORY
P <index_id> <db> <table> <index> <columns> [<fcolumns>]
<index_id>
Opened index identifier, integer, opened until the client connection
is closed
<db> Database name
<table> Table name
<index> Index name or "PRIMARY" to use primary key
<columns> List of columns to return in response, delimited by ","
[<fcolumns>] Optional argument, list of columns to filter result on
29. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL
FIND DATA | THEORY
<index_id> <op> <vlen> <v1> … <vn> [LIM] [IN] [FILTER …]
<index_id> Opened index identifier
<op> Comparison operation to use: =, >, >=, <, <=
<vlen> Number parameters to compare, equal or less than number of
columns in opened index
<v1> Value to compare using <op> with corresponding column from
index
[LIM] <limit> <offset> default values: limit=1, offset=0
[IN] @ <icol> <ivlen> <iv1> ... <ivn>
[FILTER …] <ftyp> <fop> <fcol> <fval>
30. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL
RESPONSE | THEORY
<errorcode> <numcolumns> <r1> ... <rn>
<errorcode>
Request has successfully executed or not.
'0' means success. Non-zero means an error
<numcolumns> Number of columns of the result set
<r1> … <rn> Result set, n equals <numcolumns>
31. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL
RESPONSE | THEORY
„If <errorcode> is non-zero, <numcolumns>
is always 1 and <r1> indicates a
human-readable error message, though
sometimes <r1> is not provided.”
32. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL
SQL TO HANDLERSOCKET | THEORY
SELECT value FROM t WHERE id=1 LIMIT 1
P 1 test t PRIMARY value
= 0 1
1 = 1 1 0 1 600a5841de
34. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL
PROS | SUMMARY
• Stable
• High performance
• Data consistency
• Replication support
• Not dependent on data storage engine
• Shipped with MariaDB, Percona Server
• Still allows to use SQL client/server protocol
35. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL
CONS | SUMMARY
• Filtering only using indexes
• Non-standard protocol
• Very simple authentication (password per port)
• Lack of support for transactions
• Small community
• Rarely used on production
36. HANDLERSOCKET – A NOSQL APPROACH TO MYSQL
KNOWN ISSUES | SUMMARY
• Persistent connections
• Too many indexes (> 1024?)
• Concurrent structure changes via SQL and data
modification via HandlerSocket