MariaDB is a community-developed fork of MySQL that aims to be a drop-in replacement. It focuses on being compatible, stable with no regressions, and feature-enhanced compared to MySQL. The presentation covered MariaDB's architecture including connections, query caching, storage engines, and tools for administration and development like mysql, mysqldump, and EXPLAIN.

1. MariaDB: in-depth
Colin Charles, Monty Program Ab
colin@mariadb.org | byte@bytebot.net
http://montyprogram.com/ | http://mariadb.org/
http://bytebot.net/blog/ | @bytebot on Twitter
Partner Training in Seoul, South Korea
19 March 2013
1

2. whoami
• Work on MariaDB at Monty Program Ab
• Previously from MySQL AB/Sun
Microsystems
• Have hacked on The Fedora Project and
OpenOffice.org
2

3. who is he
• Daniel Saito
• APAC Sales Director, SkySQL
• Former MySQL AB/Sun Microsystems/
Oracle
• Former NTT Docomo
3

4. Agenda
• 10.10am - we start
• 12.30pm - 1.20pm - lunch
• 1.30pm - we resume
• 6pm - makgeoli time! (I’m not paying :P)
• WiFi network is: 3Com (no password)
• Please put mobiles on vibrate
4

5. We’re very social
• Let’s use social media for the last time
today (till the break!) - come say hi now!
• facebook: fb.com/MariaDB.dbms
• twitter: @MariaDB
• g+: plus.google.com/+mariadb/
5

6. What is MariaDB?
• Community developed • MariaDB doesn’t depend
on MySQL for
• maria-captains: 50% MP, development - many
features are developed
50% community including
Sphinxsearch, Twitter, independently of MySQL
SkySQL, Taobao, Facebook,
Percona, Codership & • Backwards compatible with
more MySQL
• Feature enhanced • feature complete
• better MySQL • replication supported for
easy migration
6

7. Aims of MariaDB
• Compatible, drop-in replacement to MySQL
• your application shouldn’t care that its
running MariaDB, easy upgrade (uninstall
mysql, install mariadb, continue ops!)
• Stable (bug-free) releases with no
regressions
• GPLv2
7

8. 3 years, many server
releases
• MariaDB 5.1, GA February 2010
• MariaDB 5.2, GA November 2010
• MariaDB 5.3, GA February 2012
• MariaDB 5.5, GA April 2012
• MariaDB Galera Cluster, GA March 2013
• MariaDB 10.0, alpha November 2012, release
aim April 2013
8

9. 9

10. MariaDB 5.6?
• SkySQL is fully committed to supporting
MySQL 5.6 and all future GA releases of
the server
• For customers, SkySQL may create a
branch of MySQL 5.6 with appropriate fixes
& features from MariaDB if Oracle does
not provide timely fixes
10

11. MariaDB 10.0
11

12. The MariaDB
Ecosystem
12

13. Importance of understanding
MariaDB (and MySQL)
• Generics are inefficient
• Since you have chosen MariaDB:
• maximise its strengths
• minimise its weaknesses
13

14. Sample databases to
play with
• http://dev.mysql.com/doc/index-other.html
• sakila sample database, world
database (used in MySQL training),
menagerie database (used in book:
Beginning MySQL), employees
database (large dataset, comes with data -
best to play with)
14

15. Picking hardware
• Just use 64-bit hardware
• VM’s are improving to use 64-bit OSes &
software
• Physical > virtual
• Disk: battery backed storage, plan for RAID
usage
15

16. Testing MySQL
• Use MySQL Sandbox
• http://mysqlsandbox.net/
• Express one-click MySQL installs
• make_sandbox foo.tar.gz
• Does not require root privileges
• It is version aware:
• http://code.openark.org/blog/mysql/cheating-
mysqlsandbox-to-install-mariadb-10-0
16

17. Installation
• Binaries (tarballs) are available at http://mariadb.org/ (source too)
• Built, tested by MariaDB
• Graphical installer & configuration for Microsoft Windows,
with HeidiSQL GUI
• Up-to-date monthly release schedule
• RPM, DEB packages are provided, including YUM & APT
repositories
• Use the repository configuration tool
• Inside Linux/*BSD distributions
• Easy to install, basic defaults, may be older than upstream
17

18. Upgrades
• Review changelogs carefully, even for minor
versions
• Make backups (using xtrabackup)
• Don’t forget to run mysql_upgrade
• by default with a distribution package
• Replication can reduce downtime by upgrading
the slave, promoting it, then upgrading the
master
18

19. Packages (RPM
example)
• MariaDB-common • MariaDB-client
• character sets • mysql client utilities
• MariaDB-shared • MariaDB-test
• latest libmysqlclient, some • test suite
plugins, /etc/my.cnf.d/
• MariaDB-devel
• MariaDB-compat
• development headers enabling
• compatible client libraries that you to build your own package
replace mysql-libs
• MariaDB-CassandraSE
• MariaDB-server
• plugin for CassandraSE
• The actual server!
19

20. What is libmysqlclient?
• Client libraries
• Many applications are compiled against
libmysqlclient
• It is an Application Binary Interface (ABI)
• When you write in Java & use Connector/J,
Connector/J is compiled against libmysqlclient
• Similarly with PHP & Connector/PHP
20

21. MySQL utilities
• Why does MariaDB use all the same client utilities?
• Because the aim is to be a 100% compatible drop-in
replacement
• There are very few non-MySQL utilities shipped:
aria_chk, aria_dump_log, aria_ftdump,
aria_pack, aria_read_log
• xtstat (PBXT) is deprecated in MariaDB 5.5
• mytop as it adds features not-present upstream
21

22. mysql
• Command line tool, just shell, works over
SSH
• Allows execution of statements, keeps
track of settings, etc.
22

23. mysql
mysql -uroot -p
Enter password:
Welcome to the MariaDB monitor. Commands end with ; or g.
Your MariaDB connection id is 11
Server version: 5.5.29-MariaDB Source distribution
Copyright (c) 2000, 2012, Oracle, Monty Program Ab and others.
Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.
MariaDB [(none)]> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
+--------------------+
3 rows in set (0.00 sec)
23

24. Finding help
• How do you know what CLI options to use?
• Use the man(ual) pages!
• man mysql
• man <command_name> usually works for
any CLI command
• HELP SELECT in mysql works too (help
tables) - HELP <operator>
24

25. mysql_secure_insta
llation
• Now that you’ve tested MySQL works fine (by
default, no root password), it’s time to ensure you
secure the installation
• Set root password
• Remove anonymous users
• Disallow remote root login
• Remove test database
• Reload privileges
25

26. mysqladmin
• Manage user accounts, passwords,
permissions
• Display mysqld settings & status counters
• Shutdown, create, drop databases PROCESSLIST
SHOW
shell> mysqladmin processlist
shell> mysqladmin extended-status SHOW STATUS
shell> mysqladmin -uroot -p create wordpress
26

27. mysqldump
• Dump schema and/or data to .sql file, tab
• Useful for backups, transferring data
• Locks for consistency, so troublesome
shell> mysqldump --all-databases > backup.sql
shell> mysqldump --all-databases --single-transaction >
backup.sql
27

28. Other tools
• There are many other command line tools...
• Front-ends
• HeidiSQL
• MySQL Workbench
• Webyog’s SQLyog
• Sequel Pro (OSX)
28

29. MariaDB Architecture
29

30. Making client
connections
• TCP/IP is available on all platforms
• disable using --skip-networking
• Socket files available on Unix
• fastest communication path
• MySQL connections are generally cheap
• If you have many short running queries (typical web
app), use MariaDB threadpool
• Set connection limit max_connections=n in my.cnf
30

31. Connection Pool
• mysqld’s main thread listens for connections
• Each connection has a thread assigned to it
• Threads can be:
• created newly
• reused from thread cache
• User authentication processed based on hostname,
username & password
• Client specific buffers for session variables are allocated
31

32. Query Cache
• Stores SELECT queries and their result sets
• Subquery cache exists only in MariaDB
• Frequently changed table data will cause
queries to naturally be missed
• Regularly replaced these days with
memcached, et al.
32

33. SQL Parser
• Lexical scanner & grammar rules
• Parse SQL into tokens
• Apply grammar rules to check statement
validity
• Construct a parse tree for Optimizer to
use
33

34. Optimizer
• Reads the parse tree and calculates the best
query execution plan (QEP) to handle query
• find indexes
• determine JOIN order
• eliminate unnecessary tables
• etc.
• The optimizer is smart, don’t try to force query
plans
34

35. Pluggable Storage
Engines
• MariaDB can use many storage engines
with different features
• Installed/removed on fly with INSTALL/
UNINSTALL PLUGIN
• Mixing & matching on same server, even on
same query
35

36. Value proposition
• No other database offers this capability
• Unmatched flexibility + customisation potential
• MEMORY engine for performance/routine lookup
data
• Right storage engine can improve performance in
many applications
• ARCHIVE compresses data, up to 80%
• Partners & community benefit from this
36

37. What makes engines
different?
• Storage: how the data is stored on disk
• Or in NDB (memory+disk), CassandraSE (access a Cassandra Cluster),
SphinxSE (access the Sphinx daemon)
• Indexes: improves search operations
• Memory usage: improves data access for speed
• Transactions: protects the integrity of your data (Atomic-Consistent-Isolated-
Durable - ACID)
• Locking level: MyISAM (table locks), InnoDB (row locks), old BDB (page locks)
• Data types: Data types may be converted, MEMORY doesn’t support TEXT, etc.
• Caching: InnoDB caches data & indexes, MyISAM caches indexes only (relying on OS
disk cache for data)
• Full-text search capability: MyISAM has this, InnoDB 5.6 just got this
• GIS: MyISAM & Aria work (R-tree indexes exist), InnoDB 5.7 will probably have this
37

38. INFORMATION_SCHE
MA
• Holds metadata (data about the data) on all
other databases & tables, exposed as
regular tables
• Generated on the fly
• Has extensions in MariaDB as we expose
more data than native MySQL
38

39. PERFORMANCE_SCH
EMA
• Allows for monitoring execution at a low
level
• This is a storage engine, monitoring server
events (anything that takes time & can be
instrumented)
• Tables are views or temporary tables that
use no on-disk storage
• Such instrumentation isn’t free
39

40. MySQL 101
• q / ctrl+D works • no grants, no access
• if statement is not terminated by • ERROR 1046 (3D000): No
“;” (or uncommonly, g) and you hit database selected <-- only in
Enter, you get another line for input databases can you make tables
->
• CREATE TABLE foo (name
• G - vertical column output VARCHAR(20), sex
CHAR(1), birth DATE);
• SHOW is a MySQL-ism
• CREATE DATABASE foo;
• SHOW TABLES;
• USE foo;
• DESCRIBE foo;
• GRANT ALL on test.* TO
‘dbuser’@‘localhost’;
40

41. MySQL 101
• INSERT INTO foo • SELECT name FROM
VALUES foo WHERE
(‘fossasia’,’F’, sex=’f’;
‘2010-11-12’);
• SELECT * FROM
foo;
• SELECT name FROM
foo;
41

42. Use EXPLAIN
42

43. Ranges
• SELECT * from room WHERE room_date
between ‘2011-06-27’ AND ‘2011-07-02’;
• must ensure that index is available on field
being operated on by range operator
(room_date)
• too many records to return? Range
optimisation won’t be used, and you get a
index or full table scan
43

44. Scans & seeks
• A seek, jumps into a random place (on
disk/in memory) to fetch needed data
• A scan, jumps to the start of data, and
sequentially reads (on disk/in memory) until
the end of needed data
• Large amounts of data? Scan operations
probably better than large amount of seeks
44

45. When do you get a full
table scan
• No WHERE condition
• No index on any field with a WHERE
condition
• When range returns a large number of
rows
• SELECT * FROM
45

46. Primary keys
• Always add one
• When using InnoDB, use a small value (it is
appended to every record in a secondary
index)
• Don’t add a PK? InnoDB adds one by
default for you, and its a 6-byte integer
46

47. Schema/data types
• MySQL has 9 to represent integers
• Integer: TINYINT, SMALLINT,
MEDIUMINT, INT, BIGINT
• Floating point: FLOAT, DOUBLE
• Fixed point: DECIMAL
• Other: BIT
47

48. Bad schema design
• INT(1)
• this is NOT 1 digit. This is 4-bytes in size. A
TINYINT is 1 byte
• TINYINT UNSIGNED stores 0-255 values
• BIT? If values are 0 or 1, can be used
48

49. Bad schema design
• BIGINT AUTO_INCREMENT
• INT UNSIGNED stores 4.3 billion values!
• If you have a billion rows, think about
PARTITIONing already
49

50. Not using UNSIGNED
• Don’t want negative values?
• Use UNSIGNED
• Adds level of data integrity for cases where
negative values might be present
50

51. CHAR(n)
• This is what VARCHAR was built for
• VARCHAR(n) works
• Imagine storing simply 10 bytes in
CHAR(128)? Wasted space
51

52. ONLINE ALTER
• In MySQL 5.6 (and upcoming MariaDB 10)
• 5.5 have external tools:
• http://openarkkit.googlecode.com/svn/trunk/openarkkit/doc/
html/oak-online-alter-table.html
• https://www.facebook.com/note.php?note_id=430801045932
• http://www.percona.com/doc/percona-toolkit/2.1/pt-online-
schema-change.html
• So, be very careful when you use ENUM columns
• Probably only useful for sex (‘M’, ‘F’) or status (‘A’, ‘I’), since “Active”
and “Inactive” is something you do in the presentation layer
• Internally stored as 0 or 1 (very efficient)
52

53. Storing IPv4/6
addresses
• INT UNSIGNED works
• VARCHAR(15) results in an average of 12
bytes being used
• ... versus 4 bytes via INT UNSIGNED
• IPv6 have data types - INET6_ATON(expr) &
INET6_NTOA(expr)
• https://mariadb.atlassian.net/browse/
MDEV-274
53

54. Using NOT NULL
• Can save a byte per column per row data,
depending on the storage engine being used
• Always define columns as NOT NULL
unless there is a reason not to
54

55. Character sets
• Default is UTF-8
• The web is UTF-8
• You can define this at the database, schema, table or
even column level
• MySQL internal buffers are fixed width
• VARCHAR(255) utf8 is 765 bytes just to store 1
byte!
• Efficient disk usage with VARCHAR, but internal
memory usage isn’t
55

56. Tuning tips
• 1,000 small queries is worse than 1 slow
query!
• Bytes? We save space because you want
things in contiguous amounts of memory
56

57. Let’s go back to storage
engines for a moment
57

58. Using different engines
• Create a table with a specified engine
• CREATE TABLE t1 (..) ENGINE=XtraDB;
• Changing existing tables
• ALTER TABLE t1 ENGINE=Aria;
• SHOW ENGINES;
58

59. Transactional vs. non-
transactional
• Transaction-safe tables • Update fails? Changes
(InnoDB) have advantages reverted
over non-transaction safe
tables (MyISAM): • Concurrency - tables w/
many update +
• server crash? Automatic concurrent reads
recovery, or a backup
+transaction log • Disadvantages in today’s
environments (transaction
• ROLLBACK can be overhead = slower), more
executed to ignore disk space requirements,
changes more memory to perform
updates don’t seem like they
apply any longer
59

60. Indexes
• Tree Indexes • Hash Indexes (MEMORY, NDB,
InnoDB)
• B-Trees
• If table fits entirely in
• B+Trees (InnoDB) memory, fastest way to
perform queries is a hash
• T-Trees (NDB) index
• Red-black binary trees • InnoDB has an internal
(MEMORY) adaptive hash index. InnoDB
monitors index searches,
• R-Trees (MyISAM for spatial and if it notices that it will
benefit from a hash index,
indexes)
InnoDB automatically builds
one. (5.1.24 and greater)
60

61. Does the storage engine
really make a difference?
MyISAM Inserts XtraDB Inserts ARCHIVE Inserts
User Load Per Second Per Second Per Second
1 3,203.00 2,670.00 3,576.00
4 9,123.00 5,280.00 11,038.00
8 9,361.00 5,044.00 13,202.00
16 8,957.00 4,424.00 13,066.00
32 8,470.00 3,934.00 12,921.00
64 8,382.00 3,541.00 12,571.00
Using mysqlslap, against MariaDB 5.1.55, the ARCHIVE
engine has 50% more INSERT throughput compared to
MyISAM, and 255% more than XtraDB.
61

62. MyISAM
• Pros? • no transactions
• excellent INSERT • no foreign key
performance support
• small footprint • Typical uses
• supports full-text • logging
search (FTS)
• auditing
• Cons?
• data warehousing
62

63. MyISAM II
• In my.cnf, remember to • myisam_use_mmap
set the key_buffer_size. enables MyISAM to use
This is memory*0.40, as memory mapping (7-40%
MyISAM uses the OS speed improvement)
cache for tables
• key_cache_segments = 1
• Using DRBD (which enables segmented key
requires transactional caches in MariaDB -
engine) but still want ~250% improvements, as
FTS? Create a FTS it mitigates thread
“slave” MyISAM table contention for key cache
lock
63

64. InnoDB
• Maintains its own buffer pool (does
aggressive memory caching)
• Uses tablespaces (several files on disk, raw
disk support)
• Typically used for OLTP operations
64

65. InnoDB: Tuning
• innodb_file_per_table - • innodb_flush_log_at_trx
split InnoDB data into a _commit=1 - logs
file per table, rather than flushed to disk at each
one contiguous file transaction commit.
Guarantees ACID.
• allows optimize table
`table` to run • innodb_log_file_size -
keep it high
• innodb_buffer_pool_size (64-512MB), though
recovery time increases
=(memory*0.8)
(largest=4GB)
65

66. InnoDB: Tuning II
• SHOW ENGINE INNODB STATUS;
• InnoDB supports both row & statement
based replication
• Assign a primary key, otherwise InnoDB
assigns one for you. Also keep it small - its
appended to every record in secondary
index
66

67. ARCHIVE
• Store large amounts of data without
indexes, in small disk footprint
• SELECT and INSERT operations only
• Good for data audit use
• Uses AZIO (zlib) compression
67

68. FederatedX
• Create logical pointers to tables that exist on other MySQL servers;
these can then be linked together to form one logical database
• A federated table pointing to an InnoDB table on another server, will
have transaction support (in 5.1)
• Capabilities limited to underlying engine on remote server
• CREATE TABLE t1 (...) ENGINE=FEDERATED
CONNECTION='mysql://username:pwd@myhost:3306/db_name/
tbl_name
• Can also be used for synchronous replication
• Federated table on master server pointing to slave; triggers on
master table to write all changes to remote table once applied to
the master
68

69. Memory
• Previously known as HEAP tables
• In-memory engine
• Hash index used by default, B-Tree available too
• Typical uses?
• Lookup tables
• Session data
• Temporary tables
• Calculation tables
• Cons?
• Server dies, all rows are lost
69

70. Aria
• Based off the 5.1 code
• 1.0 – crash-safe MyISAM, with cacheable row format
• 1.5 – concurrent INSERT/SELECT
• Soon to be merged into 6.0, then...
• 2.0: transactional + ACID compliance
• 3.0: high concurrency, online backup
• Goal: ACID compliant, MVCC transactional storage engine, based on MyISAM
• Target? Data warehousing
• Uses big log files (1GB by default)
• 8K pages used by default (MyISAM uses 1K pages)
• Has group commit (MariaDB 5.2) to speed up inserts
70

71. PBXT (deprecated)
• MVCC, transactional, ACID compliant, foreign key
support
• row-level locking for updates, so maximum concurrency
• immediate notification if client processes are
deadlocked
• write-once, as it uses a log-based architecture (write
data to DB without first writing to transaction log)
• support for BLOB streaming with Blob Streaming
engine
71

72. Storage Engine API
• http://dev.mysql.com/tech-resources/
articles/creating-new-storage-engine.html
• SHOW PLUGINS;
• https://kb.askmonty.org/v/extending-create-
table
• storage/example/ha_example.cc and
storage/example/ha_example.h
72

73. Writing your own
• Find the plugin path - show variables like “%plugins%”;
+---------------+-----------------------------------+
| Variable_name | Value |
+---------------+-----------------------------------+
| plugin_dir | /opt/maria/5.2.6/lib/mysql/plugin |
+---------------+-----------------------------------+
• note that this is also where you store UDFs
• Copy the relevant engine (eg. myengine.so)
• INSTALL PLUGIN myengine SONAME 'myengine.so';
• Server registers plugin to mysql.plugin table, and now ENGINE=myengine will
work
73

74. Things to think about
• Backup is not engine-independent
• MyISAM, InnoDB, PBXT
• LVM/ZFS snapshots solve this
• Different engines have different monitoring
options
• Mix and match; use summary tables
74

75. What others are using
• Wordpress (blog): uses default engine, MyISAM is fine
• MediaWiki (wiki): prefers InnoDB, except for
“searchindex” table, which is MyISAM
• http://svn.wikimedia.org/viewvc/mediawiki/trunk/
phase3/maintenance/tables.sql?view=markup
• vBulletin (forum): MyISAM
• SugarCRM (CRM): MyISAM (with conversion script to
InnoDB provided)
• Zimbra Collaboration Suite: InnoDB
75

76. Now, let’s focus on
MariaDB features
76

77. We start with...
• What came in 5.1, 5.2, 5.3, 5.5 (jump
around appropriately)
• What comes in 10.0 series
• I won’t talk about deprecated features like
OQGRAPH, PBXT in-depth, or even the
old MariaDB 5.1 pool of threads (5.5
threadpool is better)
77

78. XtraDB
• A more performant
InnoDB designed to
scale on modern
hardware
• Less checkpointing
(smoother), less flushing
to disk
78

79. Switching between
XtraDB & InnoDB
mysqld --ignore-builtin-innodb --plugin-
load=innodb=ha_innodb.so --plugin_dir=/usr/local/
mysql/lib/mysql/plugin
Or in my.cnf
[mysqld]
ignore-builtin-innodb
plugin-load=innodb=ha_innodb.so
plugin_dir=/usr/local/mysql/lib/mysql/plugin
79

80. MariaDB 5.5: an
opensource threadpool
• Modified from 5.1 (libevent • thread_handling=po
based), great for CPU ol-of-threads
bound loads and short
running queries
• https://kb.askmonty.org/en/
thread-pool-in-mariadb-55/
• Windows (threadpool),
Linux (epoll), Solaris (event
ports), FreeBSD/OSX
(kevents)
• No minimization of
concurrent transactions
with dynamic pool size
80

81. Better for DBAs: non-
blocking client library
• start operation, do work • https://
in thread, operation kb.askmonty.org/en/
processed, result travels about-non-blocking-
back operation-in-the-
client-library/
• use cases: multiple
queries against single • fast node.js driver
server (utilize more available: mariasql
CPUs); queries against
multiple servers
(SHOW STATUS on
• https://
kb.askmonty.org/en/
many machines) mariasql-for-nodejs/
81

82. LIMIT ROWS
EXAMINED
• The purpose of this
optimization is to
• SELECT * from
t1, t2 LIMIT 10
provide the means to ROWS EXAMINED
terminate the execution 1000;
of SELECTstatements
which examine too
many rows, and thus use • https://kb.askmonty.org/
en/limit-rows-examined/
too many resources.
82

83. SHOW STATUS
• SHOW STATUS provides server status
information. It is like mysqladmin extended-status.
• SHOW STATUS LIKE ‘Key%’;
• https://kb.askmonty.org/en/show-status/
• https://kb.askmonty.org/en/server-status-
variables/
• MariaDB has opened_views, executed_triggers,
executed_events, feature_* as new options
83

84. SQL Error Logging
Plugin
• Log errors sent to clients in a log file that
can be analysed later. Log file can be rotated
(recommended)
• a MYSQL_AUDIT_PLUGIN
install plugin SQL_ERROR_LOG
soname 'sql_errlog.so';
84

85. Replication: selective
skipping
• All changes that are logged as events in the
binlog are replicated to all slaves
• However, sometimes you want all to be logged
to binlog but skipped replication to slaves
• @@skip_replication (session only)
• replicate_events_marked_for_skip
= replicate|filter_on_slave|
filter_on_master (dynamic)
85

86. Replication: dynamic
variables
• The variables replicate_do_*,
replicate_ignore_*, and replicate_wild_*
have been made dynamic, so they can be
changed without requiring a server restart.
• https://kb.askmonty.org/en/dynamic-
replication-variables/
86

87. Replication: Annotation
of RBR events
• MariaDB supports statement & row based
replication (RBR)
• In RBR, the binlog has no SQL statements,
only events are logged (INSERT, DELETE, etc)
• Option to include original SQL statement
(default OFF)
• https://kb.askmonty.org/en/
annotate_rows_log_event/
87

88. Replication: binlog
event checksums
• Backport from MySQL 5.6 (in MariaDB
5.3+)
• binlog_checksum option
• Slaves perform checksums on events
received & will stop if there is corruption
• https://kb.askmonty.org/en/binlog-event-
checksums/
88

89. Replication: group
commit in the binary log
• sync_binlog=1, • http://kb.askmonty.org/en/group-commit-
for-the-binary-log
innodb_flush_log_at_trx_commi
t=1
• SHOW STATUS LIKE 'binlog_%commits';
• https://www.facebook.com/note.php?
note_id=10150261692455933
89

90. Replication: START TRANSACTION
WITH CONSISTENT SNAPSHOT
• Works with the binlog, possible to obtain the binlog position
corresponding to a transactional snapshot of the database without
blocking any other queries.
• by-product of group commit in the binlog to view commit
ordering
• Used by the command mysqldump--single-transaction
--master-data to do a fully non-blocking backup which can be
used to provision a new slave
• Works consistently between transactions involving more than one
storage engine
• https://kb.askmonty.org/en/enhancements-for-start-transaction-
with-consistent/
90

91. GIS precise
• MySQL has OpenGIS SFS (Simple feature
access, SQL access method)
• Now, SQL with full geometry types
• ST_ prefix
• http://kb.askmonty.org/en/gis-features-
in-533
91

92. Progress reporting
• ALTER TABLE & LOAD DATA INFILE
MariaDB [mail]> alter table mail engine = maria;
Stage: 1 of 2 'copy to tmp table' 17.55% of stage done
MariaDB [mail]> select id, user, db, command, state,
-> time_ms, progress from information_schema.processlist;
+---------+-------------------+-----------+----------+
| command | state | time_ms | progress |
+---------+-------------------+-----------+----------+
| Query | copy to tmp table | 23407.131 | 17.551 |
+---------+-------------------+-----------+----------+
1 row in set (0.47 sec)
92

93. TIME_MS in
I_S.PROCESSLIST
• Extra column 'TIME_MS' has been added to
the
INFORMATION_SCHEMA.PROCESSLIST
table
• Units of milliseconds with microsecond
precision (the unit and precision of the
'TIME' column is one second).
93

94. New KILL syntax
• HARD | SOFT & USER USERNAME are
MariaDB-specific
• SOFT ensures things that may leave a table in an
inconsistent state aren’t interrupted (like REPAIR
or INDEX creation for MyISAM or Aria)
KILL [HARD | SOFT] [CONNECTION |
QUERY] [thread_id | USER user_name]
94

95. SphinxSE
95

96. The old days
• Download MySQL, including sources
• Download SphinxSE for compiling
• Download Sphinx to compile with MySQL
support
• Documented: http://www.howtoforge.com/
sphinx-as-mysql-storage-engine-sphinxse
96

97. Today
• Install sphinx from your distribution
• Install MariaDB 5.5 from your distribution
or from http://mariadb.org/
• Get started!
97

98. Getting started
mysql> INSTALL PLUGIN sphinx
SONAME 'ha_sphinx.so';
Query OK, 0 rows affected
(0.01 sec)
98

99. Another engine appears
99

100. What is SphinxSE?
• SphinxSE is just the storage engine that still
depends on the Sphinx daemon
• It doesn’t store any data itself
• Its just a built-in client to allow MariaDB to
talk to Sphinx searchd, run queries, obtain
results
• Indexing, searching is performed on Sphinx
100

101. Configure sphinx!
• /usr/local/sphinx/sphinx.conf
• Source (multiple, include mysql, with
connection info)
• Setup indexer (esp. if its on localhost) -
mem_limit, max_iops, max_iosize
• Setup searchd (where to listen to, query
log, etc.)
101

102. Use case scenarios
• Already have an existing application that
makes use of full-text-search in MyISAM?
Porting should be easier
• Have a programming language without a
native API for Sphinx? Surely there’s a
connector for MariaDB ;-)
102

103. Use case scenarios
• Results from Sphinx itself almost always
require additional work involving MariaDB
• Say to pull out text column that Sphinx
index doesn’t store
• JOIN with another table (using a different
engine)
103

104. An example
CREATE TABLE t1
(
id INTEGER UNSIGNED NOT NULL,
weight INTEGER NOT NULL,
query VARCHAR(3072) NOT NULL,
group_id INTEGER,
INDEX(query)
) ENGINE=SPHINX CONNECTION="sphinx://localhost:9312/
test";
SELECT * FROM t1 WHERE query='test it;mode=any';
104

105. Sphinx search tables
• 1st column: INTEGER UNSIGNED or
BIGINT (document ID)
• 2nd column: match weight
• 3rd column: VARCHAR or TEXT (your
query)
• Query column needs indexing, no other
column needs to be
105

106. What actually happens
• SELECT passes a Sphinx query as the query
column in the WHERE clause
• searchd returns the results
• SphinxSE translates and returns the results
to MariaDB
106

107. SHOW ENGINE
SPHINX STATUS
• Per-query & per-word statistics that searchd returns are accessible via SHOW STATUS
mysql> SHOW ENGINE SPHINX STATUS;
+--------+-------+-------------------------------------------------+
| Type | Name | Status |
+--------+-------+-------------------------------------------------+
| SPHINX | stats | total: 25, total found: 25, time: 126, words: 2 |
| SPHINX | words | sphinx:591:1256 soft:11076:15945 |
+--------+-------+-------------------------------------------------+
2 rows in set (0.00 sec)
107

108. What queries are
supported?
• Most of the Sphinx API is exposed to SphinxSE
• query, mode, sort, offset, limit, index, minid,
maxid, weights, filter, !filter, range, !range,
maxmatches, groupby, groupsort, indexweights,
comment, select
• Sphinx search modes can also be supported via
_sph attributes
• obtain value of @groupby? use ‘_sph_groupby’
108

109. Efficiency
• Allow Sphinx to perform sorting, filtering, and
slicing of result set
• ... as opposed to using WHERE, ORDER BY,
LIMIT clauses on MariaDB
• Why?
• Sphinx optimises and performs better on these
tasks
• Less data packed by searchd, and transferred and
unpacked by SphinxSE
109

110. JOINs
• Perform JOINs on a SphinxSE search table using tables from other engines
SELECT content, date_added FROM test.documents docs
-> JOIN t1 ON (docs.id=t1.id)
-> WHERE query="one document;mode=any";
+-------------------------------------+---------------------+
| content | docdate |
+-------------------------------------+---------------------+
| this is my test document number two | 2006-06-17 14:04:28 |
| this is my test document number one | 2006-06-17 14:04:28 |
+-------------------------------------+---------------------+
2 rows in set (0.00 sec)
110

111. User statistics
• Understand server activity better, identify
database loads http://kb.askmonty.org/v/
user-statistics
• Must be enabled first
• /usr/local/Cellar/mariadb/5.2.7/bin/
mysqld_safe --datadir=/usr/local/var/mysql
--userstat=1
111

112. userstats
112

113. Table Elimination
• Resolve a query without accessing some
tables query refers to
• Great for querying highly normalised data
• Basis of “anchor modelling”
• http://www.anchormodeling.com/
• SQL Server 2005/2008, Oracle 11g have it
113

114. Virtual columns
• Columns that are an expression that are
calculated on retrieval
• PERSISTENT or VIRTUAL
• Similar to MS SQL or Oracle
• https://kb.askmonty.org/en/virtual-columns/
114

115. MyISAM segmented key
caches
• Mitigates thread contention for key cache lock, with
notable performance improvements
• Key caches divided into different segments, allowing
for better key cache concurrency
• 1-64 segments
115

116. Optimizer
enchancements
• Join additions
• block nested loop joins for outer-joins, block hash joins,
Batched Key Access (BKA)
• Optimization for derived tables & views
• mergeable derived tables processed like VIEWs +
optimizer creates indexes over materialized derived
tables
• Disk access optimization
• Index Condition Pushdown (ICP), Multi-Range Read
(MRR)
116

117. Subquery optimizations
• Semi-join optimization, • Goodbye rewriting as
materialization for non- JOINs or separate
correlated IN queries, queries
subquery cache
DBT-3, 60M rows, 29GB XtraDB
117

118. 118

119. EXPLAIN, DBT-3
• INSTANT EXPLAIN
• EXPLAIN Analyzer -
https://mariadb.org/
explain_analyzer/
analyze/
• Optimizer feature
comparison matrix:
https://kb.askmonty.org/
en/optimizer-feature-
comparison-matrix/
119

120. Extended keys
• Default is extended_keys=off
• Extended Keys, introduced in MariaDB 5.5, is
an optimization which makes use of existing
components of InnoDB/XtraDB keys to
generate more efficient execution plans.
Using these components in many cases
allows the server to generate execution
plans which employ index-only look-ups.
120

121. NoSQL
• Comes with HandlerSocket
• direct access to XtraDB/InnoDB for CRUD
operations
• INSTALL PLUGIN handlersocket SONAME
'handlersocket.so';
• Dynamic columns
• create columns with dynamic content
• basically a blob with handling functions
121

122. Pluggable
authentication
• MariaDB & MySQL now uses password
authentication via pluggable auth
• Unix sockets
• PAM
• Active Directory (MySQL)
• LDAP (interested?)
• Kerberos (interested?)
122

123. Windows performance
OLTP read-only OLTP write-only
123

124. MariaDB 10.0
124

125. Why MariaDB 10.0?
• The 5.5 merge took about a year (!)
• We (MariaDB-5.5) have over 1.5 million
lines of extra code with a ~61MB diff
• We didn’t want to repeat this for 5.6
• Also, MySQL 5.6 has a lot of re-factoring,
thus loosing commit history
125

126. In a nutshell
• Built on MariaDB 5.5
• Backported features from MySQL 5.6
• New features
126

127. What about tools?
• SELECT VERSION() will • Use 9.0 for a name
return 10.0.1-MariaDB
• Lie to clients (no)
• Oops, we found a bug in
MySQL: https://
mariadb.atlassian.net/
• Disallow replication
(no)
browse/MDEV-4088 &
http://bugs.mysql.com/
bug.php?id=68187
• Use handshake packet
5.5.30-mysql-10.0.2-
MariaDB without
• Still deciding: affecting VERSION() /
@@global.version
127

128. What about tools II?
• Tools really should recognise MariaDB
version as there are already many new
features that MySQL doesn’t have
• eg. HeidiSQL supports virtual columns
(http://www.heidisql.com/forum.php?
t=8671)
128

129. Versions
Date Version Status
12 Nov
10.0.0 Alpha
2012
6 Feb
10.0.1 Alpha
2013
22 Mar
10.0.2 Beta
2013
10.0.3 GA
129

130. Backported features
(i.e. these are from MySQL 5.6)
130

131. InnoDB & XtraDB
• MariaDB 10.0 ships InnoDB from MySQL 5.6
• MariaDB 10.0.2/3 will probably have InnoDB Full Text
Search enabled
• Percona XtraDB ~ mid-2013
• no performance improvements expected, just
functionality & features
• bitmap changed page tracking so xtrabackup can do
incremental backups without scanning all InnoDB files
• XtraDB when GA can be used again
131

132. More from MySQL 5.6
• PERFORMANCE_SCHEMA
• InnoDB read-only transactions (TRANSACTION
READ ONLY)
• Optimizer:
• EXISTS-TO-IN optimization
• ORDER BY...LIMIT optimization (show only few
rows of a result set)
• CURRENT_TIMESTAMP as DEFAULT for DATETIME
columns (this is a re-implementation in MariaDB)
132

133. Only in MariaDB 10.0
the new stuff!
133

134. Multi-source replication
• Work from Taobao
• Many users partition data across many
masters... now you can replicate many
masters to a single slave
• Great for analytical queries, complete
backups, etc.
• https://kb.askmonty.org/en/multi-source-
replication/
134

135. Only in 10.0
• SHOW EXPLAIN for • Per-thread memory usage
<thread_id> (https:// (Taobao)
kb.askmonty.org/en/show-
explain/) gets the query plan
of a running statement
• I_S.PROCESSLIST has
MEMORY_USAGE &
EXAMINED_ROWS
• Faster ALTER TABLE with
unique keys for Aria &
MyISAM
• SHOW STATUS has
memory usage too
• Segmented MyISAM
keycaches (up to 64)
since MariaDB 5.2 exist
too
135

136. CassandraSE
136

137. CassandraSE
• Integration with NoSQL/Big Data DB, Apache Cassandra cluster,
seen as a storage engine to MariaDB
• Combine (join) data between Cassandra & MariaDB
• Write to Cassandra from SQL (SELECT, INSERT, UPDATE,
DELETE)
• CQL is great, but the goal is for you to just work with SQL, not
switch between CQL & SQL
• Data is mapped: rowkey, static columns, dynamic columns
• super columns aren’t supported
• No 1-1 direct map for data types (ref: https://kb.askmonty.org/en/
cassandra-storage-engine/)
137

138. LevelDB Storage Engine
• LevelDB is a key-value store, used even in
the Chrome web browser
• LevelDBSE supports single-statement
transactions, secondary indexes, crash-
proof slave replication, hot backups & more
138

139. Engine-independent
persistent statistics
• InnoDB has persistent statistics in MySQL
5.6; we have an engine-independent version
• These statistics aren’t limited by the SE
API, and are used by query optimizer to
choose best execution plan for each
statement
• Statistics collected for non-indexed
columns too (unlike InnoDB’s)
139

140. Dynamic columns
• Store a different set of columns for every
row in the table (kinda NoSQL-like)
• Database interoperability is now a major
feature; CassandraSE makes uses of this
• Can now request a row in JSON format
• And yes, you can name columns finally ;)
140

141. What about MySQL
5.6?
• We love the fact that many features we’ve worked on for
a long time are now in 5.6
• Optimizer enhancements
• Microseconds
• Binary log annotations
• Binary log group commit (10.0 has a newer faster version
now)
• Precise GIS
• Threadpool
141

142. What are we missing
from 5.6 currently?
• EXPLAIN output in JSON
• Online operations (ALTER TABLE)
• InnoDB FTS
• Global transaction IDs
• InnoDB memcached interface
• Many will be ready in final 10.0 release
142

143. Support
• Five years from every release
143

144. Benchmarks
• “Lies, damned lies, and statistics” - Mark Twain
• http://blog.mariadb.org/sysbench-oltp-
mysql-5-6-vs-mariadb-10-0/
• http://dimitrik.free.fr/blog/archives/2013/02/
mysql-performance-mysql-56-vs-mysql-55-
vs-mariadb-55.html
• Yes, we’ve gotten Oracle to notice
MariaDB :-)
144

145. Continued
commitments
• Security
• Since about a year now, we’re the go-to people for security - good
track record
• We don’t like regressions
• http://www.skysql.com/blogs/hartmut/nasty-innodb-regression-
mysql-5525
• http://www.skysql.com/blogs/kolbe/heads-no-more-query-cache-
partitioned-tables-mysql-5523
• We care about backward compatibility & introduce features carefully
• XtraDB innodb_adaptive_checkpoint=none|reflex|estimate|
keep_average (no more reflex...)
145

146. What’s next
• MariaDB Galera Cluster
• Automated failover with MHA & MariaDB
146

147. What we didn’t discuss
• Configuring MariaDB (MySQL-generics)
• use a configurator tool
• Proper backups
• NDB Cluster
• MySQL features like VIEWs, PARTITIONING, event
scheduler, etc.
Go buy a MySQL training class
• Monitoring
from SkySQL! :-)
• Optimization
147

148. Q&A / Thanks
colin@mariadb.org / byte@bytebot.net
@bytebot on twitter | http://bytebot.net/blog/
148