The slde contains an introduction to the global transaction identifiers(GTIDs) in MySQL Replication. The new protocol at re-connect, skipping transactions with GTIDS, replication filters, purging logs, backup/restore ets are covered here.

1. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
MySQL Replication:
Global Transaction Identifiers(GTIDs)
Shivji Kumar Jha
Software Developer, MySQL Replication Team

2. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Safe Harbour Statement
The following is intended to outline our general product direction. It is intended
for information purposes only, and may not be incorporated into any contract.
It is not a commitment to deliver any material, code, or functionality, and should
not be relied upon in making purchasing decisions. The development, release,
and timing of any features or functionality described for Oracle’s products
remains at the sole discretion of Oracle.

3. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Program Agenda
 Introduction to MySQL Replication
 Fail-over Base
– Global Transaction Identifiers – Automatic Positioning – Hands-On
 Under the Hood: Become a GTID Expert
– How Slave Preserves GTIDs – GTIDs Must Be Unique
 Integration With Other Features
– Seeking & Skipping – Purging Binary Logs – Restoring from Backup

4. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
What is
Replication?

5. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Replication: Copy Changes Master → Slave
 MySQL Master Server
– Changes data
– Sends changes to slave
 MySQL Slave Server
– Receives changes from master
– Applies received changes to database
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
M S

6. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Replication: Copy Changes Master → Slave
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
M M/S S
S
S
S
M
Server can be master, slave or both
Master can have multiple slaves

7. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Replication: Copy Changes Master → Slave
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
S
M
M
● Slave can only have one master
S
M
M
Slave can have multiple masters!
labs.mysql.com
labs
Yippee!

8. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Replication: Copy Changes Master → Slave
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
M/S
● Circular replication is also possible
M/S
M/S
M/S
M/S
M/S

9. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
Why Use
Replication?

10. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Why Replication? – Performance
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Read scale-out
M S
write clients read clients

11. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Why Replication? – Performance
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Read scale-out
M S
write clients read clients
More
reads?
More
slaves!

12. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Why Replication? – Performance
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Read scale-out
M S
write clients read clients
More
reads?
More
slaves!
S
S
S
M
read clients
write clients

13. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Why Replication? – Redundancy
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 If master crashes, promote slave to master
C
B
A

14. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Why Replication? – Redundancy
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 If master crashes, promote slave to master
C
B
ACrash

15. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Why Replication? – Redundancy
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 If master crashes, promote slave to master
C
B
A
B is the
new master

16. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Why Replication? – Online backup/reporting
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Expensive queries on slave(s)
M S
Regular clients
 Reports
 Big queries
 Business intelligence

17. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
CB
BAAC
Image from
www.ginkgomaps.com
Why Replication? – Long-distance Data
Distribution
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features

18. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
How Does
Replication
Work?

19. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
All Changes Written to Binary Log
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
A
binary log
Client

20. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
All Changes Written to Binary Log
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
A
binary log
Client
create table t (a int);

21. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
All Changes Written to Binary Log
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
create...
A
binary log
Client
Table t
create table t (a int);

22. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
All Changes Written to Binary Log
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
create...
A
binary log
Client
Table t
create table t (a int);
insert into t values (1);

23. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
All Changes Written to Binary Log
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
create...
insert...A
binary log
Client
Table t
1
create table t (a int);
insert into t values (1);

24. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Slave Initiates Replication
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
B
binary log
A
binary log
Client
1. Slave sends request
to start replication
to master
2. Master sends stream
of replication data
to slave

25. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Binary Log Sent to Slave, Re-executed
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
B
binary log
A
binary log
Client

26. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Binary Log Sent to Slave, Re-executed
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
create...
B
binary log
A
binary log
Client

27. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Binary Log Sent to Slave, Re-executed
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
create...
Table t
B
binary log
create...
A
binary log
Client

28. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Binary Log Sent to Slave, Re-executed
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
create...
Table t Table t
create...
B
binary log
create...
A
binary log
Client

29. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Binary Log Sent to Slave, Re-executed
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
create...
insert...
Table t Table t
create...
B
binary log
create...
A
binary log
Client

30. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Binary Log Sent to Slave, Re-executed
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
create...
insert...
Table t
1
Table t
create...
B
binary log
create...
insert...A
binary log
Client

31. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Binary Log Sent to Slave, Re-executed
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
create...
insert...
Table t
1
Table t
1
create...
insert...B
binary log
create...
insert...A
binary log
Client

32. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Makes sense?

33. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Program Agenda
 Introduction to MySQL Replication
 Fail-over Base
– Global Transaction Identifiers – Automatic Positioning – Hands-On
 Under the Hood: Become a GTID Expert
– How Slave Preserves GTIDs – GTIDs Must Be Unique
 Integration With Other Features
– Seeking & Skipping – Purging Binary Logs – Restoring from Backup

34. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Fail-over
 Crash can happen to anyone:
– Hardware failure
– Human mistake
– Bug
– Natural disaster
 Automatic fail-over with Global Transaction Identifiers:
– Reduces admin overhead
– Prevents planned downtime
– Prevents unplanned downtime
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features

35. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Fail-over
 Handle server crash with minimal disruption
 Example 1: tree
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
C
B
A

36. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Fail-over
 Handle server crash with minimal disruption
 Example 1: tree
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
C
B
ACrash

37. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Fail-over
 Handle server crash with minimal disruption
 Example 1: tree
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
C
B
A

38. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Fail-over
 Handle server crash with minimal disruption
 Example 1: tree
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
C
B
A
Make B the new master

39. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Fail-over
 Handle server crash with minimal disruption
 Example 1: tree
 Example 2: line
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features

40. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Fail-over
 Handle server crash with minimal disruption
 Example 1: tree
 Example 2: line
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
CA B

41. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Fail-over
 Handle server crash with minimal disruption
 Example 1: tree
 Example 2: line
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
CA BCrash

42. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Fail-over
 Handle server crash with minimal disruption
 Example 1: tree
 Example 2: line
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
CA B

43. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Fail-over
 Handle server crash with minimal disruption
 Example 1: tree
 Example 2: line
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
CA B
Make A direct master of C

44. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Fail-over
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Handle server crash with minimal disruption
 Example 1: tree
 Example 2: line
 Example 3: circle
C
A
B
D

45. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Fail-over
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Handle server crash with minimal disruption
 Example 1: tree
 Example 2: line
 Example 3: circle
C
A
B
DCrash

46. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Fail-over
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Handle server crash with minimal disruption
 Example 1: tree
 Example 2: line
 Example 3: circle
C
A
B
D

47. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Fail-over
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Handle server crash with minimal disruption
 Example 1: tree
 Example 2: line
 Example 3: circle
C
A
B
D
Make a shortcut
in the circle

48. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Fail-over
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Handle server crash with minimal disruption
 Example 1: tree
 Example 2: line
 Example 3: circle
 Example 4: scheduled maintenance
C
A
B
D
Take
out D!

49. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Fail-over
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Handle server crash with minimal disruption
 Example 1: tree
 Example 2: line
 Example 3: circle
 Example 4: scheduled maintenance
C
A
B
D
Maintain
D offline,
without
disrupting
service

50. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Fail-over
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Handle server crash with minimal disruption
 Example 1: tree
 Example 2: line
 Example 3: circle
 Example 4: scheduled maintenance
C
A
B
D

51. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Fail-over
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Handle server crash with minimal disruption
 Example 1: tree
 Example 2: line
 Example 3: circle
 Example 4: scheduled maintenance
 Example 5: arbitrary topology
A
B
C
D
E
F

52. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Wow!
Lets
get started with
fail-over.

53. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Program Agenda
 Introduction to MySQL Replication
 Fail-over Base
– Global Transaction Identifiers – Automatic Positioning – Hands-On
 Under the Hood: Become a GTID Expert
– How Slave Preserves GTIDs – GTIDs Must Be Unique
 Integration With Other Features
– Seeking & Skipping – Purging Binary Logs – Restoring from Backup

54. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Global Transaction Identifiers
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Server assigns Global Transaction Identifier (GTID) to every
transaction:
– server_uuid:number
a61678ba-4889-4279-9e58-45ba840af334:1
– server_uuid identifies the server; globally unique
– number is incremented by 1 for each transaction on this server
 Writes GTID to binary log
 Slave ensures transaction gets the same GTID when re-executed

55. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Global Transaction Identifiers
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
master> CREATE TABLE t1 (a INT);
master> SELECT @@global.gtid_executed;
a61678ba­4889­4279­9e58­45ba840af334:1
master> INSERT INTO t1 VALUES (1);
master> INSERT INTO t1 VALUES (2);
master> SELECT @@global.gtid_executed;
a61678ba­4889­4279­9e58­45ba840af334:1­3
Note: interval
New variable:
gtid_executed

56. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Global Transaction Identifiers
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
master> SELECT @@global.gtid_executed;
a61678ba­4889­4279­9e58­45ba840af334:1­10000
slave> SELECT @@global.gtid_executed;
a61678ba­4889­4279­9e58­45ba840af334:1­9999
Slave is missing
one transaction
Slave is missing
one transaction

57. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
But
how are GTIDs
used in
fail-over?

58. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Program Agenda
 Introduction to MySQL Replication
 Fail-over Base
– Global Transaction Identifiers – Automatic Positioning – Hands-On
 Under the Hood: Become a GTID Expert
– How Slave Preserves GTIDs – GTIDs Must Be Unique
 Integration With Other Features
– Seeking & Skipping – Purging Binary Logs – Restoring from Backup

59. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Automatic Positioning
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Master–slave protocol:
– Slave sends @@gtid_executed to master
– Master sends all other transactions to slave
id1,trx1,
id2,trx2
id1,trx1,
id2,trx2,
id3,trx3
A 2. id3, trx3, …
1. id1…id2
B

60. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Automatic Positioning
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Example 1: tree
(master)
A
id1,trx1,
id2,trx2,
id3,trx3
(slave)
C
id1,trx1
(slave)
id1,trx1,
id2,trx2B

61. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Automatic Positioning
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Example 1: tree
(master)
A
id1,trx1,
id2,trx2,
id3,trx3
(slave)
C
id1,trx1
(slave)
id1,trx1,
id2,trx2B
Crash!

62. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Automatic Positioning
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Example 1: tree
(master)
A
id1,trx1,
id2,trx2,
id3,trx3
(slave)
C
id1,trx1
(slave)
id1,trx1,
id2,trx2B

63. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Automatic Positioning
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Example 1: tree
(master)
A
id1,trx1,
id2,trx2,
id3,trx3
(slave)
C
id1,trx1
(slave)
id1,trx1,
id2,trx2B id1
id2, trx2,...
What the
slave has
What the
slave does
not have

64. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Automatic Positioning
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Example 1: tree
 Example 2: circle
BA C
client client

65. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Automatic Positioning
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Example 1: tree
 Example 2: circle
id1,trx1
BA C
client client trx1

66. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Automatic Positioning
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Example 1: tree
 Example 2: circle
id1,trx1
B
id2,trx2
A C
client clienttrx2 trx1

67. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Automatic Positioning
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Example 1: tree
 Example 2: circle
id1,trx1,
id2,trx2B
id2,trx2
A C
client clienttrx2 trx1

68. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Automatic Positioning
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Example 1: tree
 Example 2: circle
id1,trx1,
id2,trx2,
id3,trx3
B
id2,trx2
A C
client clienttrx2 trx1, trx3

69. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Automatic Positioning
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Example 1: tree
 Example 2: circle
id1,trx1,
id2,trx2,
id3,trx3
B
id2,trx2
A C
client clienttrx2 trx1, trx3
Crash!

70. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Automatic Positioning
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Example 1: tree
 Example 2: circle
id1,trx1,
id2,trx2,
id3,trx3
B
id2,trx2
A C
client clienttrx2 trx1, trx3

71. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Automatic Positioning
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Example 1: tree
 Example 2: circle
id1,trx1,
id2,trx2,
id3,trx3
B
id2,trx2
A C
client clienttrx2 trx1, trx3
id2
id1,trx1,id3,trx3,...
What the
slave has
What the
slave does
not have

72. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Awesome!
How do I set
it up?

73. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Program Agenda
 Introduction to MySQL Replication
 Fail-over Base
– Global Transaction Identifiers – Automatic Positioning – Hands-On
 Under the Hood: Become a GTID Expert
– How Slave Preserves GTIDs – GTIDs Must Be Unique
 Integration With Other Features
– Seeking & Skipping – Purging Binary Logs – Restoring from Backup

74. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Hands-On
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Prerequisites:
– Use transactional storage engine for all tables (InnoDB)
– Don't use CREATE TABLE … SELECT
– Don't execute CREATE TEMPORARY TABLE or
– DROP TEMPORARY TABLE inside a transaction

75. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Hands-On
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Prepare the server for fail-over
– Sync and stop all servers
– Add to every my.cnf:
gtid­mode=on
enforce­gtid­consistency=on
log­bin
log­slave­updates
– Start all servers
– Execute:
> CHANGE MASTER TO MASTER_AUTO_POSITION = 1

76. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Hands-On
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Perform fail-over
On the slave, simply point to the new master:
> CHANGE MASTER TO MASTER_HOST = '<host>',
MASTER_PORT = <port number>,
MASTER_USER = '<user name>'
MASTER_PASSWORD = 'secret';
 No positions needed!

77. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Hands-On
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
 Perform fail-over
C
B
A
CHANGE MASTER TO
MASTER_HOST = 'B',
MASTER_PORT = <B's port>,
MASTER_USER = '<user name>'
MASTER_PASSWORD = 'secret';

78. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Interesting!
I want to learn
more.

79. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Program Agenda
 Introduction to MySQL Replication
 Fail-over Base
– Global Transaction Identifiers – Automatic Positioning – Hands-On
 Under the Hood: Become a GTID Expert
– How Slave Preserves GTIDs – GTIDs Must Be Unique
 Integration With Other Features
– Seeking & Skipping – Purging Binary Logs – Restoring from Backup

80. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Under the Hood: How Slave Preserves GTIDs
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
B
id1,trx1,
id2,trx2A
Slave
= client

81. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Under the Hood: How Slave Preserves GTIDs
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
B
id1,trx1,
id2,trx2A
Slave
= client
SET GTID_NEXT = 'id1';
trx1

82. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Under the Hood: How Slave Preserves GTIDs
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
id1,trx1
B
id1,trx1,
id2,trx2A
Slave
= client
SET GTID_NEXT = 'id1';
trx1

83. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Under the Hood: How Slave Preserves GTIDs
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
id1,trx1
B
id1,trx1,
id2,trx2A
Slave
= client
SET GTID_NEXT = 'id1';
trx1
SET GTID_NEXT = 'id2';
trx2

84. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Under the Hood: How Slave Preserves GTIDs
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
id1,trx1,
id2,trx2B
id1,trx1,
id2,trx2A
Slave
= client
SET GTID_NEXT = 'id1';
trx1
SET GTID_NEXT = 'id2';
trx2

85. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Under the Hood: GTID_NEXT
 GTID_NEXT – session system variable
 Default: “AUTOMATIC”
– → Server generates new GTID for next transaction
 Slave thread sets to “UUID:NUMBER”
– → Server uses specified GTID for next transaction
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features

86. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Under the Hood: GTID_NEXT
 Clients can set GTID_NEXT too:
 Mysqlbinlog outputs SET GTID_NEXT statements:
id1,trx1
id2,trx2
set gtid_next = “id1”;
trx1
set gtid_next = “id2”;
trx2
mysqlbinlog
binary log
id,insert
A
Client
set gtid_next=”id”;
insert
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features

87. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Program Agenda
 Introduction to MySQL Replication
 Fail-over Base
– Global Transaction Identifiers – Automatic Positioning – Hands-On
 Under the Hood: Become a GTID Expert
– How Slave Preserves GTIDs – GTIDs Must Be Unique
 Integration With Other Features
– Seeking & Skipping – Purging Binary Logs – Restoring from Backup

88. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Under the Hood: GTIDs Must Be Unique
 Impossible to execute a GTID twice:
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
id1,trx1
id1,trx2A

89. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Under the Hood: GTIDs Must Be Unique
 Impossible to execute a GTID twice:
 Try to execute a GTID second time → transaction skipped:
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
id1,trx1
id1,trx2A
id1,trx1
A
Client

90. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Under the Hood: GTIDs Must Be Unique
 Impossible to execute a GTID twice:
 Try to execute a GTID second time → transaction skipped:
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
id1,trx1
id1,trx2A
id1,trx1
A
Client
set gtid_next=”id1”;
trx2

91. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Under the Hood: GTIDs Must Be Unique
 Impossible to execute a GTID twice:
 Try to execute a GTID second time → transaction skipped:
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
id1,trx1
id1,trx2A
id1,trx1
A
Client
set gtid_next=”id1”;
trx2
trx2 not executed!

92. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
I love
GTIDs!
Is there anything
that I should pay
attention to?

93. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Program Agenda
 Introduction to MySQL Replication
 Fail-over Base
– Global Transaction Identifiers – Automatic Positioning – Hands-On
 Under the Hood: Become a GTID Expert
– How Slave Preserves GTIDs – GTIDs Must Be Unique
 Integration With Other Features
– Seeking & Skipping – Purging Binary Logs – Restoring from Backup

94. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Skipping a Transaction
 Common use case: skip a bad transaction
 Transaction should be skipped forever
 Not enough to move replication position
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
A
id1,trx1,
id2,trx2,
id3,trx3
id1,trx1
B
Bad
transaction

95. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Skipping a Transaction
 Common use case: skip a bad transaction
 Transaction should be skipped forever
 Not enough to move replication position
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
A
id1,trx1,
id2,trx2,
id3,trx3
id1,trx1,
id3,trx3B
Bad
transaction

96. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Skipping a Transaction
 Common use case: skip a bad transaction
 Transaction should be skipped forever
 Not enough to move replication position
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
A
id1,trx1,
id2,trx2,
id3,trx3
id1,trx1,
id3,trx3B
Bad
transaction trx2 comes back
after reconnect
id2, trx2, …
id1,id3

97. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Skipping a Transaction
 Common use case: skip a bad transaction
 Transaction should be skipped forever
 Not enough to move replication position
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
A
id1,trx1,
id2,trx2,
id3,trx3
id1,trx1,
id3,trx3B
Bad
transaction trx2 comes back
after reconnect
So we don't
allow seeking!
id2, trx2, …
id1,id3

98. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Skipping a Transaction
 Recall: Try to execute a GTID second time → transaction skipped
 Interesting feature:
Client executes GTID before slave
→ slave skips transaction
 To force slave thread to skip a transaction:
slave> SET GTID_NEXT=“GTID of transaction to skip”;
slave> COMMIT;
 Transaction is skipped forever
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features

99. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Skipping a Transaction
 How to skip a
transaction forever
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
A
id1,trx1,
id2,trx2,
id3,trx3
id1,trx1
B
Bad
transaction
set gtid_next=”id2”;
commit;Client

100. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Skipping a Transaction
 How to skip a
transaction forever
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
A
id1,trx1,
id2,trx2,
id3,trx3
id1,trx1,
id2, –B
Bad
transaction
set gtid_next=”id2”;
commit;Client

101. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Skipping a Transaction
 How to skip a
transaction forever
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
A
id1,trx1,
id2,trx2,
id3,trx3
id1,trx1,
id2, – ,
id3,trx3
B
Bad
transaction
set gtid_next=”id2”;
commit;Client

102. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Skipping a Transaction
 How to skip a
transaction forever
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
A
id1,trx1,
id2,trx2,
id3,trx3
id1,trx1,
id2, – ,
id3,trx3
B
Bad
transaction
set gtid_next=”id2”;
commit;Client
id4, trx4, …
id1,id2,id3

103. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Filters
 --replicate-ignore-table etc
 Transactions should be filtered-out forever
 Slave commits empty transaction
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
A
(crashed)(master)
A
id1,trx1,
id2,trx2,
id3,trx3
(slave)
id1,trx1,
id2,-
id3,trx3
B
Filters out
trx2
Logs id2 with an
empty transaction

104. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Skipping Stuff: Summary & Recipe
 To seek forward from position A to B:
– Commit empty transaction for each GTID between A and B
 To seek backward:
– Not possible, does not make sense
 To skip N transactions:
– Get the GTIDs, commit empty transaction for each GTID
 Replication filters automatically commit empty transactions
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features

105. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
So the rule
is easy:
skip = empty
transaction

106. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Program Agenda
 Introduction to MySQL Replication
 Fail-over Base
– Global Transaction Identifiers – Automatic Positioning – Hands-On
 Under the Hood: Become a GTID Expert
– How Slave Preserves GTIDs – GTIDs Must Be Unique
 Integration With Other Features
– Seeking & Skipping – Purging Binary Logs – Restoring from Backup

107. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Purging Binary Logs
 Binary logs are rotated and can be purged
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
A
id4,trx4,
id5,trx5,
id6,trx6
master-bin.02 master-bin.03
id7,trx7,
id8,trx8
master-bin.01
id1,trx1,
id2,trx2,
id3,trx3

108. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Purging Binary Logs
 Binary logs are rotated and can be purged
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
A
id4,trx4,
id5,trx5,
id6,trx6
master-bin.02 master-bin.03
id7,trx7,
id8,trx8
Client PURGE BINARY LOGS TO 'master­bin.02'
master-bin.01
id1,trx1,
id2,trx2,
id3,trx3

109. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Purging Binary Logs
 Binary logs are rotated and can be purged
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
A
id4,trx4,
id5,trx5,
id6,trx6
master-bin.02 master-bin.03
id7,trx7,
id8,trx8
XPurged!
Client PURGE BINARY LOGS TO 'master­bin.02'
master-bin.01

110. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Purging Binary Logs
 On Master: cannot send purged GTIDs to slave
– @@gtid_purged = all purged GTIDs
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features

111. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Purging Binary Logs
 On Master: cannot send purged GTIDs to slave
– @@gtid_purged = all purged GTIDs
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
B Purged!A
id4,trx4,
id5,trx5,
id6,trx6
master-bin.02
XPurged!
id1,trx1,
id2,trx2
slave-bin.01
gtid_purged=id1,id2,id3 gtid_executed=id1,id2
master-bin.01

112. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Purging Binary Logs
 On Master: cannot send purged GTIDs to slave
– @@gtid_purged = all purged GTIDs
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
B Purged!A
id4,trx4,
id5,trx5,
id6,trx6
master-bin.02
XPurged!
id1,trx1,
id2,trx2
slave-bin.01
Error!
id1,id2
Nothing sensible to do:
B needs trx3
which is purged
gtid_purged=id1,id2,id3 gtid_executed=id1,id2
master-bin.01

113. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Purging Binary Logs
 On Slave: must send all GTIDs (purged or not) to master
– @@gtid_executed = all GTIDs (purged or not)
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
A
slave-bin.02
XPurged!A
master-bin.02
id1,trx1,
id2,trx2,
id3,trx3
id4,trx4,
id5,trx5
id4,trx4
slave-bin.01master-bin.01
id5,trx5
id1…id4
Note: slave must send id1...id3 despite purged
or else master would send trx1…trx3

114. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Purging Binary Logs
 On Master: cannot send purged GTIDs to slave
– @@gtid_purged = all purged GTIDs
 On Slave: must send all GTIDs (purged or not) to master
– @@gtid_executed = all GTIDs (purged or not)
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
A
id4,trx4,
id5,trx5,
id6,trx6
master-bin.02 master-bin.03
id7,trx7,
id8,trx8
XPurged!
@@gtid_purged = id1 … id3 @@gtid_executed = id1 … id8
master-bin.01

115. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Purged and Executed are Stored in Binary Log
 Binary logs begin with:
[ list of all GTIDs in all previous binary logs ]
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
A
[id1…id3]
id4,trx4,
id5,trx5,
id6,trx6
master-bin.02 master-bin.03
[id1…id6]
id7,trx7,
id8,trx8
XPurged!
master-bin.01
GTIDs in
master-bin.01 +
master-bin.02
GTIDs in
master-bin.01

116. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Purged and Executed are Stored in Binary Log
 Binary logs begin with:
[ list of all GTIDs in all previous binary logs ]
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
A
[id1…id3]
id4,trx4,
id5,trx5,
id6,trx6
master-bin.02 master-bin.03
[id1…id6]
id7,trx7,
id8,trx8
XPurged!
@@gtid_executed
=
head of newest log +
GTIDs in newest logid1…id3
id1…id6 + id7 + id8
master-bin.01
@@gtid_purged
=
head of oldest log

117. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Purging Binary Logs: Summary
 Works automatically, behind the scenes
 Purged GTIDs are stored in @@gtid_purged
 @@gtid_purged is a subset of @@gtid_executed
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features

118. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Automatic
is the word!
Does purging have
anything to do with
backups?

119. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Program Agenda
 Introduction to MySQL Replication
 Fail-over Base
– Global Transaction Identifiers – Automatic Positioning – Hands-On
 Under the Hood: Become a GTID Expert
– How Slave Preserves GTIDs – GTIDs Must Be Unique
 Integration With Other Features
– Seeking & Skipping – Purging Binary Logs – Restoring from Backup

120. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Provision a Slave From a Backup
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
id1,create,
id2,insertA
Table t
1
B

121. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Provision a Slave From a Backup
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
id1,create,
id2,insertA
Table t
1
B
Backup
Table t
1
Backup

122. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Provision a Slave From a Backup
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
id1,create,
id2,insertA
Table t
1
B
Table t
1Backup
Table t
1
Backup Restore

123. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Provision a Slave From a Backup
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
id1,create,
id2,insertA
Table t
1
B
Table t
1Backup
Table t
1
Backup Restore
Binary log
not
restored!

124. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Provision a Slave From a Backup
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
id1,create,
id2,insertA
Table t
1
B
Table t
1Backup
Table t
1
Backup Restore
Binary log
not
restored!
Set gtid_executed and
gtid_purged to id1,id2
(or else A would send
id1,id2 when B connects)

125. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Provision a Slave From a Backup
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features
id1,create,
id2,insertA
Table t
1
B
Table t
1Backup
Table t
1
Backup Restore
Binary log
not
restored!
Set gtid_executed and
gtid_purged to id1,id2
(or else A would send
id1,id2 when B connects)
XPurged!
set @@gtid_purged = “id1,id2”

126. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Provision a Slave From a Backup
 @@gtid_executed must agree with database state
– @@gtid_executed after restore = @@gtid_executed at backup
 @@gtid_purged = GTIDs in @@gtid_executed that are not in log
– Binary logs not (normally) restored
– @@gtid_purged after restore = @@gtid_executed after restore
 @@gtid_purged is settable:
– SET @@GLOBAL.GTID_PURGED = "GTID_EXECUTED at backup"
– Also sets @@GTID_EXECUTED to the same value
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features

127. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Restore a Backup
 Backup tools must store @@gtid_executed
 Restore tools must execute:
– SET @@GLOBAL.GTID_PURGED =
"GTID_EXECUTED at backup"
Introduction to MySQL Replication  Fail-over Base  Under the Hood  Integration With Other Features

128. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
You had
me at “GTID”
What shall I
do now?

129. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Next Steps
 More on GTIDs
– svenmysql.blogspot.co.uk
 More on MySQL 5.6 Replication
– http://dev.mysql.com/tech-resources/articles/
mysql-5.6-replication.html
 Try Out MySQL 5.6
– http://dev.mysql.com/downloads/mysql/