Mais conteúdo relacionado Semelhante a Setup & Operate Tungsten Replicator (20) Setup & Operate Tungsten Replicator2. ©Continuent 2014
Introducing Continuent
2
• The leading provider of clustering and
replication for open source DBMS
• Our Product: Continuent Tungsten
• Clustering - Commercial-grade HA, performance
scaling and data management for MySQL
• Replication - Flexible, high-performance data
movement
4. ©Continuent 2014
Tungsten Replicator Overview
4
Master
(Transactions + Metadata)
Slave
THL
DBMS
Logs
Replicator
(Transactions + Metadata)
THLReplicator
Download
transactions
via network
Apply using JDBC
5. ©Continuent 2014
Master Replication Service
5
Extract Filter Apply
Stage
Extract Filter Apply
Stage
Pipeline
MySQL
Master
Transaction
History Log
In-Memory
Queue
Slave
ReplicatorsBinlog
tcp/ip
6. ©Continuent 2014
Slave Replication Service
6
Extract Filter Apply
Stage
Extract Filter Apply
Stage
Extract Filter Apply
Stage
Pipeline
Transaction
History Log
In-Memory
Queue
Slave
DBMS
Master
Replicator
tcp/ip
7. ©Continuent 2014
EC2 Installation Using Github project
1.Set up Github vagrant project
git clone https://github.com/continuent/
continuent-vagrant
2.Pick EC2 Vagrantfile and fill in keys
3.Copy examples/MasterSlave/default.pp to
manifests/
4.Run launch.sh
7
(Follow directions in README.md)
8. ©Continuent 2014
Tpm installation overview
8
/opt/continuent/etc/tungsten/tungsten.ini
check prereqs
[copy code]
configure topology
[defaults]!
user=tungsten!
home-directory=/opt/continuent!
datasource-user=tungsten!
datasource-password=secret!
application-port=3306!
application-user=app_user!
application-password=secret!
skip-validation-check=MySQLPermissionsCheck!
start-and-report=true!
!
[east]!
master=db1!
slaves=db2,db3!
topology=master-slave!
tpm update
(Installation)
10. ©Continuent 2014
Commands to Check Replication
• trepctl services -- All services
• trepctl status -- Overall pipeline status
• trepctl status -name stages -- Stage summary
• trepctl status -name stores -- Store summary
• trepctl status -name tasks -- Task detail
• trepctl [-service name] heartbeat -- Is it Alive?
10
12. ©Continuent 2014
Reconfiguring a Replicator
• Edit /etc/tungsten/tungsten.ini
• No restart on local host
trepctl offline!
tpm update --no-restart!
trepctl configure!
trepctl online!
!
• With automatic restart
tpm update
12
Don’t change replicator configuration files directly!
14. ©Continuent 2014
Parallel Replication of Sharded Data
14
Extract Filter Apply
Stage
Extract Filter Apply
Stage
Stage
Pipeline
Transaction
History Log
Parallel
Queue
Slave
DBMS
Extract Filter Apply
Extract Filter Apply
Extract Filter Apply
Master
Replicator
tcp/ip
16. ©Continuent 2014
The Scene of the Crime
16
trepctl -host db2 status
Processing status command...
NAME VALUE
---- -----
...
pendingError : Event application failed:
seqno=1957469 fragno=0 message=java.sql.SQLException:
Statement failed on slave but succeeded on master
pendingErrorCode : NONE
pendingErrorEventId : mysql-bin.000008:0000000461736666;0
pendingErrorSeqno : 1957469
pendingExceptionMessage: java.sql.SQLException: Statement
failed on slave but succeeded on master
create table trouble(id int primary
key) /* ___SERVICE___ = [university] */
...
Finished status command...
17. ©Continuent 2014
Correcting problems
1.Look at the transaction (thl list -seqno NNN)
2.Look at the replicator message log
3.Fix the problem and go online
OR
4.Skip over the problem and go online
OR
5.Reprovision the slave
17
19. ©Continuent 2014
Simple Facts about Logs
• Replicator log = THL (Transaction History Log)
• One THL per replication service
• 2x the size of MySQL binlog
• Default retention of 7 days
• Take service offline to move or delete logs
safely
• The thl tool shows you log contents
19
24. ©Continuent 2014
What Backup Tools Does MySQL Have?
24
MySQL
Server
Process
File System
DBMS Files
mysqldump
(dataset < 10Gb)
xtrabackup
(10Gb < dataset < 1Tb
tar/rsync
(dataset < 1Tb)
file system snapshot
(1Tb << dataset)
26. ©Continuent 2014
How Do I Install Fan-In Replication?
26
db1
Replicator
db3
Service db1
db2
Replicator
Service db2
Replicator
Service db1
Service db2
27. ©Continuent 2014
How Do I install Multi-Master?
27
db1 Replicator
Service db1
Service db2
Service db3
db3
Service db1
Service db2
Service db3
db2
Replicator
Service db1
Service db2
Service db3
29. ©Continuent 2014
Tpm install from staging directory
29
db1
db2
db3
Staging copy
of files
check prereqs
copy code
configure topology
tpm configure ...
tpm update
30. ©Continuent 2014
Commands to Perform Staging Install
30
$ wget http://downloads.tungsten-replicator.org/download.php?
file=tungsten-replicator-2.2.1-403.tar.gz
!
$ tar -xvzf tungsten-replicator-2.2.1-403.tar.gz
!
$ ln -s tungsten-replicator-2.2.1-403 tungsten
!
$ tungsten/tools/tpm configure defaults --reset
--user=tungsten --home-directory=/opt/continuent
--datasource-user=tungsten --datasource-password=secret
--skip-validation-check=MySQLPermissionsCheck
--start-and-report=true
!
$ tungsten/tools/tpm configure fanin
--members=db1,db2,db3
--masters=db1,db2
--master-services=db1,db2
--topology=fan-in
!
$ tungsten/tools/tpm update
32. ©Continuent 2014
Tungsten Replicator is Open Source
32
• Project home:
http://code.google.com/p/tungsten-replicator/
• Log bugs, find builds, post in replicator discussion
group
• Documentation:
https://docs.continuent.com
35. ©Continuent 2014
More Tungsten University Webinars!
35
• Set Up and Operate Tungsten Clusters
(22 May 2014 10am PDT)
• Set Up and Operate Real-Time Data Loading
into Hadoop
(29 May 2014, 10am PDT)
• Set Up and Operate Open Source Oracle
Replication
(5 June 2014, 10am PDT)
Send feedback to: tu@continuent.com
36. ©Continuent 2014
Continuent Web Page:
http://www.continuent.com
!
Tungsten Replicator 2.0:
http://code.google.com/p/tungsten-replicator
Our Blogs:
http://scale-out-blog.blogspot.com
http://datacharmer.org/blog
http://www.continuent.com/news/blogs
http://flyingclusters.blogspot.com/
560 S. Winchester Blvd., Suite 500
San Jose, CA 95128
Tel +1 (866) 998-3642
Fax +1 (408) 668-1009
e-mail: sales@continuent.com