Galera Cluster 3.0 Features

G a l e r a C l u s t e r
Release 3 New Features
Seppo Jaakola, Alex Yurchenko
Codership Oy

2
www.codership.com
Agenda
1.  Galera Cluster for MySQL
2.  Release 3 New Features:
  WAN Replication
  5.6 GTID Support
  MySQL Replication Support
  etc.
3.  Galera Project

3
www.codership.com
MySQL MySQL MySQL
Galera Cluster
Galera Replication
< No SPOF
all nodes are equivalent and fully
representative of the cluster

4
www.codership.com
MySQL MySQL MySQL
Galera Cluster
Galera Replication
read & write
< No SPOF
< Transparent
clients connect directly to server

5
www.codership.com
MySQL MySQL MySQL
Galera Cluster
Galera Replication
read & write
< No SPOF
< Transparent
< Synchronous
transaction is either committed on
every node or not at all

6
www.codership.com
MySQL MySQL MySQL
Galera Cluster
Galera Replication
read & writeread & writeread & write
< No SPOF
< Transparent
< Multi-master
unrestricted writes to every node
< Synchronous

7
www.codership.com
Looks like a single MySQL server
with multiple connection points
Galera Cluster
read & writeread & writeread & write
< No SPOF
< Transparent
< Multi-master
unrestricted writes to every node
< Synchronous

8
www.codership.com
Also...
  Parallel slave applying
  Practically no slave lag
  Instant trivial failover
  Automatic node provisioning
  Works great in WAN

9
www.codership.com
How is that possible?
“Cluster” vs. “replication”
  Global context:
- Membership
- Global State
- Global Transaction ID

10
www.codership.com
How is that possible? Limitations:
  InnoDB only
  Primary Keys is a must
  Commit latency
  Doesn't like huge transactions (max 2GB)
  DEADLOCK on COMMIT
  ...

12
www.codership.com
Galera Cluster
DBMS
Galera plugin
wsrep hooks
wsrep API
dlopen
LOAD

13
www.codership.com
Galera Cluster
DBMS
wsrep provider
GCS framework
replication
wsrep hooks
wsrep API
dlopen
certification
vsbes gcommspread
Galera
Plugin

14
www.codership.com
Galera Cluster Releases
Galera projects:
  Universal Replication Interface (wsrep API)
  Defines the interface between DBMS and replication plugin
  Versioned by API level #1-25. Galera 3.1 is using #25
  Galera Replication Plugin
  Latest API #23 version is 2.7
  And API #25 version is 3.1
  Replication API implementation in MySQL server
  5.5.34-25.9 for MySQL 5.5
  5.6.14-25.1 for MySQL 5.6
  All open-source on Launchpad

15
www.codership.com
Replication Plugin
Replication plugin is runtime loadable
  Set global wsrep_provider=none;
  Set global wsrep_provider='/usr/lib/libgalera_smm.so';
With no replication plugin specified, server
should work as vanilla MySQL server

16
www.codership.com
20 Galera Replication Plugin Releases since 2009

17
www.codership.com
35 Galera Cluster for MySQL Releases since 2009

18
www.codership.com
Roadmap to 3.0
Demo
Release

19
www.codership.com
Roadmap to 3.0
Demo
Release
0.6 Release
  Global Transactio ID
GTID

20
www.codership.com
Roadmap to 3.0
Demo
Release
0.7 Release
  DDL support
0.6 Release
GTID

21
www.codership.com
Roadmap to 3.0
Demo
Release
0.7 Release
  DDL support
0.6 Release
GTID
0.8 Release
  Parallel replication
  SST script API

22
www.codership.com
Roadmap to 3.0
Demo
Release
0.7 Release
  DDL support
1.0 Release
  Foreign key support
  gcache
0.6 Release
GTID
0.8 Release

23
www.codership.com
Roadmap to 3.0
Demo
Release
0.7 Release
  DDL support
1.0 Release
  gcache
0.6 Release
GTID
0.8 Release
2.0 Release
  IST
  TOI & RSU

24
www.codership.com
Roadmap to 3.0
Demo
Release
0.7 Release
  DDL support
1.0 Release
  gcache
0.6 Release
GTID
0.8 Release
2.0 Release
  IST
  TOI & RSU
OMG
!!

25
www.codership.com
Three Letter Abbreviations
There is three letter abbreviation for it: TLA
We can fill the Galera terminology with 6760
acronyms
With current rate we can continue Galera
project until year 2978
http://en.wikipedia.org/wiki/Three-letter_acronym

26
www.codership.com
MySQL MySQL MySQL
Multi-Master Replication
Galera Replication
Replication is synchronous
read & write read & write read & write
Client can connect to any node
There can be several nodes
Read & write access to any node

27
www.codership.com
MySQL
Multi-Master Replication
a
Multi-master cluster looks
like one big database with
multiple entry points
read & write read & write read & write

28
www.codership.com
Galera Cluster
  Synchronous multi-master cluster
  no data loss
  no slave lag
  no slave failover
  For MySQL/InnoDB
  3 or more nodes needed for HA
  No single point of failure

29
www.codership.com
Galera Cluster
  Good Performance
  Optimistic concurrency control
  Parallel Replication
  Optimized Group Communication
  99.99% transparent
  InnoDB look & feel
  Automatic node joining
  Works in LAN / WAN / Cloud

31
www.codership.com
Galera Cluster 3
Galera Cluster 3.1 GA released Nov 12
(Galera 3.1, MySQL-wsrep 5.6.14)
Focus on:
  WAN replication
  Asynchronous replication topologies

32
www.codership.com
Galera Cluster 3 (Galera-3.x, MySQL-5.6.x)
Featuring:
  MySQL 5.6 Support:
  Support for MySQL 5.6 GTID
  Native replication can be interleaved with Galera replication
  Optimization for WAN replication:
  Cluster can be divided in segments to indicate network proximity
  New writeset format:
  Optimized certification key format
  128-bit checksums
  Direct IO cache → writeset copy.
  A number of bug fixes and minor improvements:
  Hardware-accelerated CRC32 on network packets, etc.

34
www.codership.com
MySQL 5.6 Support
Both MySQL-wsrep 5.5 and 5.6 have support
for Galera 3.x
From Galera perspective, it does not matter
much what the DBMS is like, as long as it
supports transactions.
In MySQL 5.6 interesting new features are:
-  MySQL GTID
-  InnoDB full text search
-  Overall performance improvements

35
www.codership.com
MySQL 5.6 Support
  We will also support MariaDB 10, work is ongoing.
  Percona XtraDB Cluster will also have 5.6 based
version coming soon.

36
www.codership.com
MySQL 5.6 Support
  Support for MySQL 5.1 is dropped.
  Future releases are for MySQL 5.5 and 5.6 only.
  Now can be built on Linux, Solaris, FreeBSD,
MacOS.

38
www.codership.com
GTID Support
MySQL 5.6 introduces global transaction
identifier (GTID), which can greatly ease up
the MySQL replication master fail over.
In Galera Cluster, all nodes will generate
different binlog files:
  Binlog events are same and in same order, but
binlog file offsets may vary
  Galera community has developed miracles to cope
with this, it is amazing what a single line of perl
code can do

39
www.codership.com
Galera Binlogging
Node 1
mysqld-bin.00007
INSERT INTO
t0...
Node 2
mysqld-bin.00008
INSERT INTO
t0...

40
www.codership.com
GTID Support
Galera will preserve the MySQL GTID events
both in “galera replicated mysql slave” and
“pre-ordered replication” methods.
Galera nodes will generate Galera Cluster
GTID for Mysql replication. MySQL slaves will
see the Galera Cluster as one MySQL master.
Master fail over in Galera Cluster can happen
by using the GTID.

41
www.codership.com
Galera Binlogging
Node 1
mysqld-bin.00007
INSERT INTO
t0...
Node 2
mysqld-bin.00008
INSERT INTO
t0...
mysql-GTID-1
mysql-GTID-1

42
www.codership.com
Galera Binlogging
INSERT INTO
t0...
INSERT INTO
t0...
mysql-GTID-1
mysql-GTID-1
mysql-GTID-2
mysql-GTID-2
GTID for Cluster
events
Node 1 Node 2
mysqld-bin.00007 mysqld-bin.00008

Async Replication in/out Galera Cluster

44
www.codership.com
Galera + MySQL Replication
Galera Cluster can operate both as
MySQL master and MySQL slave, but
  MySQL master fail over in Galera Cluster has
been challenging until now
  MySQL Slave operation has performance
bottleneck
Galera 3.x addresses both problems

45
www.codership.com
Galera as MySQL Slave
Galera Cluster can operate as MySQL
slave in two ways:
1.  MySQL master is like client connection to
Galera node (version 2 and 3)
2.  MySQL replication events will be delivered
as pre-ordered events interleaved in Galera
replication (new version 3 feature)

46
www.codership.com
MySQL
master
Node 1
slave
MySQL
replication
Galera as MySQL Slave
a
Node 2 Node 3

47
www.codership.com
MySQL Slave through Galera Replication
MySQL replication
Query
processing
Certification
Commit
Commit processing
Write set population
replication
WS
WS
Slave
processing
WS

48
www.codership.com
MySQL Slave through Galera Replication
MySQL replication
Query
processing
Certification
Commit
Commit processing
Write set population
replication
WS
WS
WS
Slave
processing

49
www.codership.com
Async Replication Bottleneck
  As MySQL replication events are treated as regular
client transactions, they must go through Galera
replication pipeline at commit time
  … and this adds some delay before commit
  MySQL replication is single threaded
  MySQL 5.6 “parallel replication” may help, if you
have several schemas
-  But only, if you have several schemas
-  MySQL 5.7 and MariaDB 10 will have proper parallel
replication
Galera Cluster will not apply replication as fast as native
MySQL Slave

50
www.codership.com
MySQL Replication Bundling
  Replication events can be bundled to commit as a
single group
  wsrep_mysql_replication_bundle=n
-  Groups n mysql replication transactions in one large
transaction
  Helps with the latency, you can go up to 5K trx/
sec
Less waits for replication synchronization

Galera 3 Pre-ordered Replication

52
www.codership.com
Pre-ordered replication
MySQL replication
Query
processing
Commit
replication
G
M G
Slave
processing
Preordered
replication

53
www.codership.com
Pre-ordered Replication
  MySQL replication stream constitutes partial
order, which can be interleaved with cluster
replication
  SQL slave thread broadcasts MySQL replication
events before Galera slaves apply them
  Applying does not slow down SQL slave thread
  Note, we expect MySQL replication stream to be
conflict free

54
www.codership.com
Node1
MySQL slave
Node 2
Node 3
M-5
G-2
G-4
M-3
G-1
MySQL
replication
MySQL
master1
Galera
Replication

55
www.codership.com
  Galera nodes will receive and apply MySQL
replication events in same order and interleaved
with local Galera replication events
  If MySQL replication has conflicts with Galera
events, node will do emergency shutdown
  A better approach for dealing with conflicts is under works
  The usual multi-master conflict scenarios apply here:
  Delete conflict
  Update conflict
  Uniqueness conflict

56
www.codership.com
MySQL Replication – Multi Source
Node1
MySQL slave
MySQL
replicationMySQL
master1
Node2
MySQL slave
Node3
MySQL master
MySQL
master2
MySQL
replication
MySQL
slave
MySQL
replication
Galera
Replication

57
www.codership.com
MySQL Replication – Multi Source
Works with both MySQL replication strategies
  “Slave through Galera Replication”, provides
certification for MySQL masters
-  Consistency is guaranteed
  “Pre-ordered Replication”, is high speed but
requires consistency guarantee from application

59
WAN replication
  Works fine
  Use higher timeouts
  No impact on reads
  No impact within a transaction
  adds 0-300 ms to commit latency

60
www.codership.com
MySQL MySQL
WAN Cluster
a Galera Replication
Reads & writes
All super fast
COMMIT
Delay between 1-1.5 RTT

61
WAN replication
  Works fine
  Use higher timeouts
  No impact on reads
  No impact within a transaction
  adds 0-300 ms to commit latency
  No major impact on tps
  Quorum between data centers
-  3 data centers
-  Weighted quorum calculation
-  Garbd arbitrator

62
www.codership.com
WAN Replication 2.0
Galera 3.0 has new replication mode, which is
optimized for WAN networks (or any network
with high latencies in general).
But, let's first take a look how Galera 2.0
replication looks between data centers.

63
www.codership.com
A-1
WAN Replication 2.0
A-2 A-3
B-1
B-2 B-3
Data center A Data center B
  All point-to-point connections will be needed for replication

64
www.codership.com
A-1
WAN Replication 2.0
A-2 A-3
B-1
B-2 B-3

65
www.codership.com
A-1
WAN Replication 2.0
A-2 A-3
B-1
B-2 B-3
  I said ALL

66
www.codership.com
WAN Replication 2.0
Due to using all point-to-point connections,
the transaction latency is dictated by the
longest latency in the cluster replication
mesh
Despite of this anomaly, Galera Cluster 2.0
and earlier have been widely used in WAN
environments

67
www.codership.com
A-1
WAN Replication 3.0
A-2
B-1
B-3
  Data messages between cluster segments is sent only once
  Replication events will be distributed within each segment p2p
B-2A-3

68
www.codership.com
A-1
WAN Replication 3.0
A-2
B-1
B-3
  Replication between segments go over one link only
  Replication events will be distributed within each segment p2p
  Segment gateways can change per transaction
B-2A-3

69
www.codership.com
A-1
WAN Replication 3.0
A-2 A-3
B-1
B-2 B-3
WS
commit

70
www.codership.com
A-1
WAN Replication 3.0
A-2 A-3
B-1
B-2 B-3
WS WS
WS

71
www.codership.com
A-1
WAN Replication 3.0
A-2 A-3
B-1
B-2 B-3
WS WS WS WS

72
www.codership.com
WAN Replication 3.0
Define cluster segments up front by node
location
gmcast.segment = 1..255

73
www.codership.com
SST in WAN
Galera will choose SST/IST Donor from same
segment, if possible

75
www.codership.com
New writeset format: Certification Keys
Writeset contains references for all affected
primary, unique and foreign keys.
These affected keys are also stored in
certification index maintained in each node
to find out possible multi-master conflicts.

76
www.codership.com
Keys
Binlog
events
Keys
Binlog
events
Writeset
Certification index
Galera 2.x Galera 3.x
conversion pointer
Writeset
Certification index

77
www.codership.com
Galera 3.0 has refactored these key
representations so that same compact format
can be used in write set and certification
index.
  Memory footprint is minimal for each key and
fixed length.
  Certification check runs faster on the new format.
  More and larger transactions can run
simultaneously without memory issues.

78
www.codership.com
New Writeset Format
In our tests we see
  5-10% less CPU usage on both master and slave
sides
  10-15% better throughput with big transactions
  2x smaller memory overhead
Compared to the latest 2.x release (25.2.8)

79
www.codership.com
Huge Transaction Support
Writeset format refactoring is one step
forward in huge transaction support.
However, more work remains for achieving
that:
  We still replicate transaction as a singe batch at
commit time. It needs to be fragmented.
LOAD DATA transaction can be split in a
series of 10K row transactions
wsrep_load_data_splitting = ON | OFF

81
www.codership.com
Galera Project
  Galera Cluster for MySQL
  6 years development
  based on MySQL server community edition
  Fully open source
  Active community
  ~3 releases per year
  Latest GA releases: 2.8, 3.1

83
www.codership.com
Galera Cluster
MySQL
Community
edition

84
www.codership.com
WSREP patch
Galera Cluster
MySQL
Community
edition
Galera Cluster for
MySQL

85
www.codership.com
WSREP patch
Galera Project
MySQL
Community
edition
Galera WSREP provider
Galera Cluster for
MySQL
wsrep API (dlopen)

86
www.codership.com
WSREP patch
Galera Project
MySQL
Community
edition
Percona
Server
MariaDBforkfork
Galera Cluster for
MySQL
wsrep API (dlopen)

87
www.codership.com
WSREP patchWSREP patch
Galera Project
Percona
Server
MariaDB
merge mergeWSREP patch
MySQL
Community
edition
Galera Cluster for
MySQL
wsrep API (dlopen)

88
www.codership.com
Galera Project
Percona XtraDB
Cluster
MariaDB Galera
Cluster
WSREP patchWSREP patch
Percona
Server
MariaDB
merge mergeWSREP patch
MySQL
Community
edition
Galera Cluster for
MySQL
wsrep API (dlopen)

89
www.codership.com
Codership
  Consulting and support services for Galera
  Technology and support partners:
  Percona
  SkySQL
  Monty Program
  Severalnines
  FromDual
  Capside

Questions?
Thank you for listening!
Happy Clustering :-)

Galera Cluster 3.0 Features

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Galera Cluster 3.0 Features

Semelhante a Galera Cluster 3.0 Features (20)

Último

Último (20)

Galera Cluster 3.0 Features