1. MySQL Proxy:
a MySQL toolbox
Architecture and
Concepts of Misuse
Jan Kneschke
MySQL Enterprise Tools
mailto: jan@mysql.com
2. What is MySQL Proxy
Started Feb 2007
Current: MySQL Proxy 0.7.0
Source available on launchpad.net
$ bzr branch lp:mysql-proxy
Foundation of the MEM 2.0 Agent
Fully support for MySQL Enterprise
Customers
2 (+ 1) Developers, 1 QA
3. How it looks for a bird
Inspecting traffic
Logging, auditing
Routing connections and queries
Rewriting queries
Injection queries
“Be a good man in the middle”
4. How it looks for a frog
MySQL Proxy
Lua Scripting layer
Proxy Plugin Admin Plugin
Network Core (libmysql-proxy)
libmysql libevent libmysql liblua
-chassis -proto
5. The libs - chassis
Command Line Interface
Config-File support
Logging (event-log, syslog, stderr,
logfiles)
Plugin Loading
Threading
6. The libs - Protocols
Documented …
http://forge.mysql.com/wiki/MySQL_Internals_Clie
MySQL Client Protocol
Binary Logs
MyISAM Row Format for RBR
Hides the “creativity” of the
implementation
7. What you don't want to know
How to encode a 1234 (0x04d2) ?
In resultsets: 04 31 32 33 34
In Prepared depends on type: d2 04 00 00
In the rest of the protocol: 02 d2 04
Protocol is using a random mix of those
encodings
8. Proof of Concept
mysql-binlog-dump
Similar to mysqlbinlog
First implementation of RBR decoding
mysql-myisam-dump
Decodes .frm and .myd files
Uncovers a little gem: screens
9. Let's wrap it
Lua
Small and fast
Easy to read and write
Easy to embed
All the libs are nicely wrapped
10. The core – libmysql-proxy
Protocol live-cycle (auth, old-auth, query)
Provides the plugin hooks
Global Lua scope
Network interface
MySQL Protocol encoding/decoding
11. Lua iterator for binlogs
Expose binlog events into lua
Reading and writing
local binlog = require(“mysql.binlog”)
local f = binlog.open(“jan-binlog.log”)
for event in f:next() do
if event.type == “QUERY_EVENT” then
print(event.query.query)
end
end
12. Filtering
Filter statements from a binlog
Remove or rewrite Queries
INSERT INTO answers
VALUES ( 42, “the Answer” )
UPDATE answers
SET answer = “Thanks for the Fish”
WHERE id = 2
ALTER TABLE answers
ADD INDEX (answer);
DROP TABLE answers;
13. Merging binlogs
independent Masters like shards
Merge on timestamp
14. Druckbetankung
Use RBR to push data into a server
RBR is already in the native format
mysql-data-source
Convert input data to a live RBR-binlog-stream
mysqld
CHANGE MASTER TO … our data-source
START SLAVE
15. Slave Accelerator
Problem: Replication is synchronized,
single thread
Solution: move replication out of the
slave
Don't pull, push
Parallel connections
16. mysql-slave-accel
Read data from master
Maintain slave-state centrally
Rules to detect if data is independent
e.g. one connection per schema
Works transparently for SBR, tricky for
RBR
17. Master Failover - Classic
Master on stand-by
On failure, CHANGE MASTER on
all slaves
Promote the best slave
19. mysql-master-failover
Track active and passive master
Only forward data received from both
masters
Track matching binlog-file and -position
Active dies
Trigger switch over
Slaves get a continuos binlog stream
States
Synched, Single, Out-of-Sync
20. Thanks
Code is on launchpad.net
http://launchpad.net/mysql-proxy/
Questions to: jan@mysql.com
weigon in #mysql-proxy on
irc.freenode.net