1. <Insert Picture Here>
MySQL Cluster 7.1
Andrew Morgan Mat Keep
MySQL Cluster Product Manager MySQL Product Management
andrew.morgan@oracle.com mat.keep@oracle.com
2. Agenda
• Introduction to MySQL Cluster
• Deep Dive into MySQL Cluster 7.1 New
Features
• Resources to Get Started
3. Serving Key Markets & Industry Leaders
Web / Web 2.0 OEM / ISV's
On Demand, SaaS, Hosting Telecommunications Enterprise 2.0
Open-source powers the Web & the Network
4. MySQL Cluster Users and Applications
HA, Real Time Services: Web & Telecoms
• Subscriber Databases
• Service Delivery Platforms
• Application Servers
• Telco Value Added Services
• Web Session Stores
• eCommerce
• VoIP, IPTV & VoD
• Mobile Content Delivery
• On-Line app stores and portals
• DNS/DHCP for Broadband
• Payment Gateways
• Data Store for LDAP Directories
•http://www.mysql.com/customers/cluster/
5. MySQL Cluster -
Key Advantages
High Throughput Distributed, Parallel architecture
Reads & Writes
Transactional, ACID-compliant relational database
Carrier-Grade Shared-nothing design, synchronous data replication
Availability
Sub-second failover & self-healing recovery
Real-Time High-load, real-time performance
Responsiveness Predictable low latency, bounded access times
Incrementally scale out, scale up and scale on-line
On-Line, Linear
Scalability Linearly scale with distribution awareness
Low TCO, GPL & Commercial editions, scale on COTS
Open platform Flexible APIs: SQL, C++, Java, OpenJPA, LDAP & HTTP
6. MySQL Cluster Architecture
Parallel Database with no SPOF: High Read & Write Performance & 99.999%
uptime
Clients
MySQL Cluster Application Nodes
MySQL MySQL
Cluster Cluster
Mgmt Mgmt
MySQL Cluster Data Nodes
7. MySQL Cluster 7.0 Momentum (GA April ‘09)
Leading Provider of Mobile, Fixed & 250% Increase in Download Rate
Broadband Services across Europe & Asia
“Telenor has found MySQL Cluster to be the
best performing database in the world for
our applications.” Windows Port
Peter Eriksson, Manager, Network Provisioning
LDAP Interface
I/O Multi-Threading
2x Faster Record Handling
Largest Entertainment Guide on the
On-Line Cluster Scaling Mobile Web:
“go2 has the ability to efficiently scale
4x Higher Throughput our platform with the growth of the
mobile web”
Dan Smith, Co-Founder & CEO, go2 Media
8. MySQL Cluster CGE 7.1 –
Key Enhancements
Delivering up to 10x higher
Reducing Cost of Operations Java Throughput
Simplified Management &
MySQL Cluster Connector for
Monitoring:
Java:
NDBINFO
Native Java API
MySQL Cluster Manager (part of CGE
only) OpenJPA Plug-In
Faster Restarts
9. ndbinfo
• New database (ndbinfo) which
mysql> use ndbinfo presents real-time metric data in
mysql> show tables;
the form of tables
+-------------------+
| Tables_in_ndbinfo | • Exposes new information together
+-------------------+ with providing a simpler, more
| blocks | consistent way to access existing
| config_params |
| counters |
data
| logbuffers | • Examples include:
| logspaces | • Resource usage (memory, buffers)
| memoryusage |
• Event counters (such as number of
| nodes |
READ operations since last restart)
| resources |
| transporters | • Data node status and connection status
+-------------------+
10. ndbinfo
• Example 1: Check memory usage/availability
mysql> select * from memoryusage;
+---------+--------------+------+------+
| node_id | DATA_MEMORY | used | max |
+---------+--------------+------+------+
| 3 | DATA_MEMORY | 594 | 2560 |
| 4 | DATA_MEMORY | 594 | 2560 |
| 3 | INDEX_MEMORY | 124 | 2336 |
| 4 | INDEX_MEMORY | 124 | 2336 |
+---------+--------------+------+------+
• Note that there is a DATA_MEMORY and INDEX_MEMORY row for each data node
in the cluster
• If the Cluster is nearing the configured limit then increase the DataMemory and/or
IndexMemory parameters in config.ini and then perform a rolling restart
11. ndbinfo
• Example 2: Check how many table scans performed on each data node since the last
restart
mysql> select node_id as 'data node', val as 'Table Scans' from counters
where counter_name='TABLE_SCANS';
+-----------+-------------+
| data node | Table Scans |
+-----------+-------------+
| 3 | 3 |
| 4 | 4 |
+-----------+-------------+
• You might check this if your database performance is lower than anticipated
• If this figure is rising faster than you expected then examine your application to
understand why there are so many table scans
12. MySQL Cluster 7.1: ndbinfo
• Example 3: Check if approaching the point at which the undo log completely fills up
between local checkpoints (which could result in delayed transactions or even a database
halt if not addressed):
mysql> select node_id as 'data node', total as 'configured undo log buffer
size', used as 'used buffer space' from logbuffers where log_type='DD-
UNDO';
+-----------+---------------------------------+-------------------+
| data node | configured undo log buffer size | used buffer space |
+-----------+---------------------------------+-------------------+
| 3 | 2096128 | 0 |
| 4 | 2096128 | 0 |
+-----------+---------------------------------+-------------------+
• If log buffer is almost full then increase size of log buffer
14. MySQL Cluster 7.1: ndbinfo.logbuffers
• View created from hidden tables to present
log buffer usage
• log_type=REDO: „total‟ represents the
amount of memory configured for the redo
mysql> select * from ndbinfo.logbuffers; logs and „used‟ indicates how much of it is
+---------+----------+--------+----------+----------+--------+ currently being used. If the buffer fills up
| node_id | log_type | log_id | log_part | total | used |
+---------+----------+--------+----------+----------+--------+ then will see 1221 “REDO log buffers
| 3 | REDO | 0 | 0 | 67108864 | 262144 |
| 3 | DD-UNDO | 4 | 0 | 2096128 | 0 | overloaded” errors – avoid this by
| 4 | REDO | 0 | 0 | 67108864 | 262144 | increasing the RedoBuffer configuration
| 4 | DD-UNDO | 4 | 0 | 2096128 | 0 |
+---------+----------+--------+----------+----------+--------+ parameter if „used‟ approaches „total‟
• log_type=DD_UNDO: „total‟ represents the
amount of memory configured for the undo
logs (UNDO_BUFFER_SIZE when creating
the log group) and „used‟ indicates how
much of it is currently being used. If buffers
fill too quickly then transactions can be
delayed while they are flushed to disk; in
extreme cases causing database to halt
15. MySQL Cluster 7.1: ndbinfo.logspaces
• View created from hidden tables to
present logspace usage
• log_type=REDO: 1 row for each of the 4
mysql> select * from logspaces;
file sets for each data node.
+---------+----------+--------+----------+-----------+--------+ „total‟=NoOfFragmentLogFiles*Fragment
| node_id | log_type | log_id | log_part | total | used |
+---------+----------+--------+----------+-----------+--------+
LogFileSize and „used‟ is the amount
| 3 | REDO | 0 | 0 | 536870912 | 0 | actually used. If the files fill up before a
| 3 | REDO | 0 | 1 | 536870912 | 0 |
| 3 | REDO | 0 | 2 | 536870912 | 0 | local checkpoint can complete then error
| 3 | REDO | 0 | 3 | 536870912 | 0 | code 410 (Out of log file space
| 3 | DD-UNDO | 4 | 0 | 78643200 | 169408 |
| 4 | REDO | 0 | 0 | 536870912 | 0 | temporarily) will be observed. Avoid this
| 4 | REDO | 0 | 1 | 536870912 | 0 | by increasing NoOfFragmentLogFiles
| 4 | REDO | 0 | 2 | 536870912 | 0 |
| 4 | REDO | 0 | 3 | 536870912 | 0 | and/or FragmentLogFileSize if used
| 4 | DD-UNDO | 4 | 0 | 78643200 | 169408 |
+---------+----------+--------+----------+-----------+--------+ approaches total.
• log_type=DD_UNDO: „total‟=cumulative
size of all of the undo log files assigned
to the log group, as added using
create/alter logfile group or the
InitialLogFileGroup parameter. Add extra
undo files if „used‟ approaches „total‟ to
avoid 1501 errors
16. MySQL Cluster 7.1:
ndbinfo.memoryusage
• View created from hidden tables to present
memory limits and usage
mysql> select * from memoryusage; • Compare the amount of memory and index
+---------+--------------+------+------+ used to the amount configured for each data
| node_id | memory_type | used | max | node
+---------+--------------+------+------+
| 3 | DATA_MEMORY | 632 | 3144 | • DATA_MEMORY: „max‟ = DataMemory
| 4 | DATA_MEMORY | 632 | 3144 | parameter. If „used‟ approaches „max‟ then
| 3 | INDEX_MEMORY | 38 | 2336 | increase DataMemory and/or delete obsolete
| 4 | INDEX_MEMORY | 38 | 2336 |
rows run OPTIMIZE TABLE for the Cluster
+---------+--------------+------+------+
tables
• INDEX_MEMORY: „max‟ = IndexMemory
parameter. If „used‟ approaches „max‟ then
increase IndexMemory (+ DataMemory if
needed) and/or delete obsolete rows run
OPTIMIZE TABLE for the Cluster tables
17. MySQL Cluster 7.1: ndbinfo.nodes
• View created from hidden tables for the
status of data nodes.
mysql> select * from nodes; • For each data node that is running in the
+---------+--------+---------+-------------+ cluster, a corresponding row in this table
| node_id | uptime | status | start_phase |
+---------+--------+---------+-------------+ provides the
| 3 | 3719 | STARTED | 0 | • Uptime: time in seconds that this node has been
| 4 | 3719 | STARTED | 0 | running since it was last started or restarted.
+---------+--------+---------+-------------+ • Status: One of NOTHING, CMVMI, STARTING,
STARTED, SINGLEUSER, STOPPING_1,
STOPPING_2, STOPPING_3, or STOPPING_4.
• Start-phase: If Status=STARTING then the start-phase
(as seen through ndb_mgm), 0 otherwise.
18. MySQL Cluster Connector for Java
• New Domain Object Model
Persistence API (ClusterJ) :
• Java API
• High performance, low latency
• Feature rich
• JPA interface built upon this new
Network Java layer:
• Java Persistence API compliant
• Implemented as an OpenJPA plugin
• Uses ClusterJ where possible, reverts
to JDBC for some operations
• Higher performance than JDBC
• More natural for most Java designers
• Easier Cluster adoption for web
Data Nodes applications
19. ClusterJ
• High Performance, Easy to Use
• In the style of Hibernate / JPA / JDO
• Domain Object Model DataMapper pattern
• Data is represented as domain objects
• Domain objects are separate from business logic
• Domain objects are mapped to database tables
• Built on ndbjtie
• JNI adapter
• integral part of MySQL Cluster
• Straight mapping of MySQL Cluster API (a.k.a Ndb API) to Java
• Does not support relationships
• Look at JDO / JPA for these modelling patterns
22. ClusterJ Annotated interface example
@PersistenceCapable(table="employee")
@Index(name="idx_uhash")
public interface Employee {
@PrimaryKey
CREATE TABLE employee ( int getId();
void setId(int id);
id INT NOT NULL PRIMARY KEY,
String getFirst();
first VARCHAR(64) DEFAULT NULL,
void setFirst(String first);
last VARCHAR(64) DEFAULT NULL,
String getLast();
municipality VARCHAR(64) DEFAULT NULL, void setLast(String last);
started VARCHAR(64) DEFAULT NULL, @Column(name="municipality")
ended VARCHAR(64) DEFAULT NULL, @Index(name="idx_municipality")
department INT NOT NULL DEFAULT 1, String getCity();
UNIQUE KEY idx_u_hash (first,last) void setCity(String city)
USING HASH, String getStarted();
KEY idx_municipality (municipality) void setStarted(String date);
) ENGINE=NDBCLUSTER; String getEnded();
void setEnded(String date);
Integer getDepartment();
void setDepartment(
Integer department);}
23. ClusterJ Application code example
// Create a session (connection to the database)
SessionFactory factory = ClusterJHelper.getSessionFactory(props);
Session session = factory.getSession();
// Create and initialise an Employee
Employee newEmployee = session.newInstance(Employee.class);
newEmployee.setId(988);
newEmployee.setFirst("John");
newEmployee.setLast("Jones");
newEmployee.setStarted("1 February 2009");
newEmployee.setDepartment(666);
// Write the Employee to the database
session.persist(newEmployee);
// Fetch the Employee from the database
Employee theEmployee = session.find(Employee.class, 988);
// Make some changes to the Employee & write back to the database
theEmployee.setDepartment(777);
theEmployee.setCity("London");
session.updatePersistent(theEmployee);
// Retrieve the set all of Employees in department 777
QueryBuilder builder = session.getQueryBuilder();
QueryDomainType<Employee> domain = builder.createQueryDefinition(Employee.class);
domain.where(domain.get("department").equal(domain.param("department")));
Query<Employee> query = session.createQuery(domain);
query.setParameter("department",777);
List<Employee> results = query.getResultList();
24. ClusterJPA
• Removes ClusterJ limitations:
• Persistent classes
• Relationships
• Joins in queries
• Lazy loading
• Table and index creation from object model
• Implemented as an OpenJPA plugin
• Better JPA performance for insert, update, delete
25. ClusterJPA Annotated class example
@Entity(name = "department") mysql> describe department;
public class Department { +----------+--------------+------+-----+---------+-------+
private int Id; | Field | Type | Null | Key | Default | Extra |
+----------+--------------+------+-----+---------+-------+
private String Site; | id | int(11) | NO | PRI | NULL | |
| location | varchar(255) | YES | | NULL | |
+----------+--------------+------+-----+---------+-------+
public Department(){}
@Id public int getId()
{return Id;}
public void setId(int id)
{Id=id;}
@Column(name="location")
public String getSite()
{return Site;}
public void setSite(String
site)
{Site=site;}
}
26. ClusterJPA Application code example
EntityManagerFactory entityManagerFactory =
Persistence.createEntityManagerFactory("clusterdb");
EntityManager em = entityManagerFactory.createEntityManager();
EntityTransaction userTransaction = em.getTransaction();
userTransaction.begin();
Department dept = new Department();
dept.setId(101);
dept.setSite(“London”);
em.persist(dept);
userTransaction.commit();
userTransaction.begin();
Department theDepartment = em.find(Department.class, 101);
theDepartment.setSite(“Edinburgh”);
userTransaction.commit();
em.close();
entityManagerFactory.close();
28. MySQL Cluster Manager 1.0 Features
Automated Management Monitoring HA Operations
Cluster-Wide Status Monitoring & Disk Persistence
Management Recovery Configuration
Process Management Consistency
On-Line Operations HA Agent Operation
(Upgrades /
Reconfiguration)
29. Terms used by MySQL Cluster
Manager
• Site: the set of physical hosts which are to run
Cluster processes to be managed by MySQL
Cluster Manager. A site can include 1 or more
clusters.
• Cluster: represents a MySQL Cluster
deployment. A Cluster contains 1 or more
processes running on 1 or more hosts
• Host: Physical machine, running the MySQL
Cluster Manager agent
• Agent: The MySQL Cluster Manager process
running on each host
• Process: an individual MySQL Cluster node;
one of: ndb_mgmd, ndbd, ndbmtd, mysqld &
ndbapi*
• Package: A copy of a MySQL Cluster
installation directory as downloaded from
mysql.com, stored on each host
*ndbapi is a special case, representing a slot for an external application
process to connect to the cluster using the NDB API
30. Example configuration
mysql
client
• MySQL Cluster Manager agent runs
on each physical host
7. mysqld 8. mysqld
• No central process for Cluster
1. ndb_mgmd 2. ndb_mgmd
Manager – agents co-operate, each
one responsible for its local nodes
agent agent
• Agents are responsible for managing
192.168.0.10 192.168.0.11 all nodes in the cluster
• Management responsibilities
3. ndbd 4. ndbd
• Starting, stopping & restarting nodes
5. ndbd 6. ndbd
• Configuration changes
agent agent
• Upgrades
192.168.0.12 192.168.0.13 • Host & Node status reporting
• Recovering failed nodes
n. mysqld MySQL Server (ID=n)
n. ndb_mgmd Management Node (ID=n)
n. ndbd Data Node (ID=n)
agent MySQL Cluster Manager agent
31. Installing, configuring, running &
accessing MySQL Cluster Manager
• The agent must be installed and run on each
host in the Cluster:
mysql
1. Expand the tar-ball into a known directory
client (/usr/local/mcm)
2. Copy /usr/local/mcm/etc/mysql-cluster-manager.ini to
/home/billy/mcm and edit:
[mysql-proxy]
plugins=manager
manager-port = :1862
log-file = mysql-manager-agent.log
log-level = message
manager-directory=/home/billy/mcm/manager
agent agent
3. “plugins=manager” should always be used
4. Launch the agent process:
192.168.0.10 192.168.0.11 /usr/local/mcm/bin/mysql-cluster-manager --defaults-
file=/home/billy/mcm/mysql-cluster-manager.ini
5. Access any of the agents from any machine (that has the
mysql client installed):
agent agent C:Usersam233268>mysql -h 192.168.0.10 -P 1862 -u admin –p
Enter password: super
Welcome to the MySQL monitor. Commands end with ; or g.
Your MySQL connection id is 1
192.168.0.12 192.168.0.13 Server version: 1.0.1-agent-manager MySQL Cluster Manager
Type 'help;' or 'h' for help. Type 'c' to clear the
current input statement.
mysql>
32. Creating & Starting a Cluster
mysql
client 1. Define the site:
Mysql> create site --hosts=192.168.0.10,192.168.0.11,
-> 192.168.0.12,192.168.0.13 mysite;
2. Expand the MySQL Cluster tar-ball(s)
7. mysqld 8. mysqld
from mysql.com to known directory
1. ndb_mgmd 2. ndb_mgmd
3. Define the package(s):
agent agent Mysql> add package --basedir=/usr/local/mysql_6_3_26 6.3;
Mysql> add package --basedir=/usr/local/mysql_7_0_7 7.0;
Note that the basedir should match the
192.168.0.10 192.168.0.11
directory used in Step 2.
3. ndbd 4. ndbd
4. Create the Cluster
Mysql> create cluster --package=6.3
5. ndbd 6. ndbd -> --processhosts=ndb_mgmd@192.168.0.10,ndb_mgmd@192.168.0.11,
-> ndbd@192.168.0.12,ndbd@192.168.0.13, ndbd@192.168.0.12,
-> ndbd@192.168.0.13,mysqld@192.168.9.10,mysqld@192.168.9.11
agent agent
-> mycluster;
This is where you define what
192.168.0.12 192.168.0.13
nodes/processes make up the Cluster and
where they should run
5. Start the Cluster:
Mysql> start cluster mycluster;
33. Check the status of the Cluster
mysql
client
1. Request the status of the Cluster
processes
mysql> show status --process mycluster;
•+------+----------+--------------+---------+-----------+
7. mysqld 8. mysqld | Id | Process | Host | Status | Nodegroup |
+------+----------+--------------+---------+-----------+
| 1 | ndb_mgmd | 192.168.0.10 | running | |
1. ndb_mgmd 2. ndb_mgmd | 2 | ndb_mgmd | 192.168.0.11 | running | |
| 3 | ndbd | 192.168.0.12 | running | 0 |
agent agent | 4 | ndbd | 192.168.0.13 | running | 0 |
| 5 | ndbd | 192.168.0.12 | running | 1 |
| 6 | ndbd | 192.168.0.13 | running | 1 |
| 7 | mysqld | 192.168.0.10 | running | |
192.168.0.10 192.168.0.11 | 8 | mysqld | 192.168.0.11 | running | |
+------+----------+--------------+---------+-----------+
3. ndbd 4. ndbd
2. Get the status of the hosts & agents
mysql> list hosts mysite;
5. ndbd 6. ndbd •+--------------+-----------+---------+
| Host | Status | Version |
agent agent +--------------+-----------+---------+
| 192.168.0.10 | Available | 1.0.1 |
| 192.168.0.11 | Available | 1.0.1 |
| 192.168.0.12 | Available | 1.0.1 |
192.168.0.12 192.168.0.13 | 192.168.0.13 | Available | 1.0.1 |
+--------------+-----------+---------+
35. Setting Cluster parameters
mysql
client
• Example: Turn of privilege checking for all
MySQL Servers and change the port for
connecting to the mysqld with ID = 8 to
7. mysqld 8. mysqld 3307. Allow data nodes to be automatically
1. ndb_mgmd 2. ndb_mgmd
restarted after they fail:
mysql> set skip_grant_tables:mysqld=true,port:mysqld:8=3307,
agent agent --> StopOnError:ndbd=false mycluster;
MySQL Cluster Manager automatically
192.168.0.10 192.168.0.11
determines which nodes (processes) need
to be restarted and in which order to make
3. ndbd 4. ndbd
the change take effect but avoid loss of
5. ndbd 6. ndbd
service
agent agent
192.168.0.12 192.168.0.13
36. Upgrade Cluster
mysql
client
• Upgrade from MySQL Cluster 6.3.26 to
7. mysqld 8. mysqld 7.0.7:
1. ndb_mgmd 2. ndb_mgmd
mysql> upgrade cluster --package=7.0 mycluster;
agent agent
• Automatically upgrades each node and
192.168.0.10 192.168.0.11 restarts the process – in the correct order to
avoid any loss of service
3. ndbd 4. ndbd • Without MySQL Cluster Manager, the
5. ndbd 6. ndbd administrator must stop each process in
turn, start the process with the new version
agent agent
and wait for the node to restart before
192.168.0.12 192.168.0.13
moving onto the next one
37. "MySQL Cluster 7.1 offers the potential for users of the
FreeRADIUS AAA server to enhance operational efficiency and
performance in the management and access of their business
critical and time sensitive applications”
Alan Dekok
FreeRADIUS Project Founder, CEO of Network RADIUS
"Many of our users rely on OpenLDAP to run their most critical
directory services, and with the addition of MySQL Cluster at the
back-end, can support highly dynamic and scalable directory
databases. The release of MySQL Cluster 7.1 can enhance
manageability and platform choice for our communities and
customers."
Marty Heyman,
CEO of Symas Corporation and OpenLDAP Project
Member
38. Resources to Get Started
•MySQL Cluster 7.1, Architecture and New Features
• http://www.mysql.com/why-mysql/white-
papers/mysql_wp_cluster7_architecture.php
• MySQL Cluster Manager white paper
• http://www.mysql.com/why-mysql/white-papers/mysql_wp_cluster_manager.php
•MySQL Cluster Connector for Java white paper
• http://www.mysql.com/why-mysql/white-
papers/mysql_wp_cluster_connector_for_java.php
•MySQL Cluster 7.1 Evaluation Guide
• http://www.mysql.com/why-mysql/white-papers/mysql_cluster_eval_guide.php
•Getting Started with MySQL Cluster
• http://www.mysql.com/products/database/cluster/get-started.html
•MySQL Cluster on the Web
• http://www.mysql.com/products/database/cluster/