Introduction to MySQL, and its features with an explanation of the various processes that should be followed in order to have an efficient MySQL implementation.

1. MySQL Features
By Rakesh Kumar
OSSCube , Delhi
rakesh@osscube.com

2. Introduction

3. What we will discuss in next 1 hour
• General Info.
• MySql Enterprise
• Other Features of MySQL
> MySQL Network Monitoring and Advisory services
> Backup and Recovery
> Audit Trail
> High Availability
> Migration Tool Kit

4. General Info - Overview of MySQL AB
• Founded in 1995
• Operations in 24 countries
• World’s Most Popular Open Source Database
> Over 10 million installations
> 50,000 downloads each day
• Reduces TCO of database management
> Reliable, high performance, easy to use
• Bundled by over 100 software & hardware companies
• Partners and Telecom Service Integrators
HP, Dell, Novell, Red Hat, Unisys, Accenture, LogicaCMG, T-Systems and IPL

5. Key Features
• Written in C, C++
• Multilayered design with Independent
modules
• Fully multi-threaded using Kernel Threads
• Provides Transactional and Non-transactional
storage engines

6. Key Features Cont…
• In memory heap table support
• Server available in client server model, or
embedded DB.
• Security and privilege management
• Supports connection over SSL
• Large database, large tables (default up to
256 TB)

7. Key features contd…
• Connectors for application connectivity
• 64 Index per table, 1-16 columns per index
• Built-in tools for query analysis and space
analysis
• Localization
• Many more…

8. What is MySQL Enterprise?
MySQL Enterprise is the leading open source database solution behind
today’s modern enterprise applications. It includes:
• The MySQL Enterprise Server – the most reliable, secure, and up-to-date
version of the world’s most popular open source database.
• MySQL Network Monitoring and Advisory Services – services to ensure
the maximum quality, uptime, and performance of MySQL Enterprise
systems.
• Production Support – around-the-clock support for the MySQL
Enterprise server, along with consultative services and self-help
knowledge aids.
Designed for production use and web businesses /
enteprise customers relying on MySQL for key
business applications.

9. MySQL Enterprise vs. Community Server
Open Source
Benefits from testing by worldwide Community
Feature Releases Experimental Mature
Monthly Rapid Update Service Pack Program
Quarterly Service Pack Program
Hot Fix Program
Bug Escalation Privilege
24x7 Production Support
Platform / Application / Training Certified
Extended End-of-Life Policy
Enterprise Installer and Configuration Manager
Out-of-the-Box Configured for Enterprise use
Monitoring and Advisory Service
Automated Software Update and Alert Service
Online Knowledge Base
Extensive internal/external (Coverity, KLOCwork) testing
Benchmarked with 2,000+ connections
Legal indemnification

10. What is the MySQL Community
Server?
The MySQL Community Server is the rapidly evolving, leading edge and non-
commercially supported version of the MySQL server.
• Leading edge: new and experimental features with every release.
• Open source development and innovation. GPL license.
• High community participation. Contributions encouraged!
• Great for open source developers, technology enthusiasts, and those
wanting to experiment and try out MySQL.
• Serves as the foundation & proving ground for MySQL Enterprise.
• No formal QA, support, or certifications
Recommended for non-production applications and
those wishing to try out MySQL. Supported by the
large MySQL open source community.

11. Other Features of MySQL
• MySql Network Monitoring and Advisory services
• Audit Trail
• Backup and Recovery
• High Availability – Replication and Clusters
• Data migration from other RDBMS to MySQL

12. MySQL Network Monitoring and Advisory Service
• Single, consolidated view into entire MySQL environment
• Customizable rules-based monitoring and alerts
• Identifies problems before they occur
• Reduces risk of downtime
• Makes it easier
to scale-out without
requiring more DBAs
“Virtual MySQL DBA”
Assistant

13. Top Features & Benefits
Enterprise Dashboard
Primary benefits
• Provides “at a glance” view into health, security, availability and
performance of all MySQL servers.
• Saves DBA time writing, maintaining and monitoring
scripts that collect, compile, and report similar metrics.
• Assists DBAs in determining “where” and “how” to
spend their limited time.
Why it’s important
• Makes it easy to scale-out on MySQL without adding
more DBAs.

14. Top Features & Benefits
MySQL Network Advisors
Security Schema Administration Performance Replication Custom
• Protects MySQL • Helps DBA • Helps DBA • Makes •Makes • Allows customer
Servers design better better manage suggestions for to create their
• Uncovers databases database improving •Identifies own Advisor
Security • Uncovers processes database speed collection
loopholes • Suggests • Identifies • Provides for
Security
improvements potential custom best
loopholes
for smoother performance practice
operations bottlenecks enforcement
Each contains a set of Advisor Rules designed to enforce MySQL Best
Practices for the specific Advisor category.
Provide “Virtual MySQL DBA” expertise

15. Top Features & Benefits
MySQL Network Advisors and Rules
Primary benefits
• 100+ Advisor Rules proactively monitor 600+ MySQL and OS specific
metrics for problems before they occur.
• Extend DBA skill set to include MySQL expertise.
• Find problems/tuning opportunities DBAs cannot find
themselves.
Why they are important
• Reduce downtime and performance issues by identifying
problems and corrective actions before costly outages
occur.
Provide “Virtual MySQL DBA” expertise

16. Top Features & Benefits
Customizable Monitors & Alerts
Primary Benefits
• Saves DBA time and effort by allowing DBAs to group and monitor many
servers as single entity.
• User defined Advisor rules allow for Enterprise specific monitoring and
alerts.
• Notifications (SMTP/SNMP) can be tailored for individual or group
recipients.
Why it’s important
• Allows for customized SLAs across large
MySQL deployments.

17. Value Prop of MySQL Enterprise
Gold
Silver Silver +
Basic + • Monitors and
• Monitors all servers Optimizes Memory G
• Proactive Usage
Notifications • Replication
• Expert Advice Monitoring
Basic • Optimizes MySQL • Ensures proper
r e m t s uC ot e u a V
• Enterprise Server Configuration Replication
• Ensures settings configuration •Ensures Performance
l
Uptime/Reliability • Ensures • Rapid Problem
• Regular Service Packs Recoverability Resolution •Guards against
• Self-help Support • Tightens Security • (Hot Fix builds)
• Production Support • Extends Support •Optimizes/Tunes
• Support! Support! • Custom
o
Queries
• Database
Design

18. MySQL Network Monitoring and
Advisory Service How it works
Automated Data Collection Automated Data Collection
“Query Cache Hit Rate”
600 + MySQL and OS specific metrics
Data Evaluation
• Metrics from all SHOW commands -Default or custom expression evaluation
“Query Cache Has Sub-optimal Hit Rate”
• Metrics from MySQL provided
Thresholds
scripts -Violation?
“Query Cache Hit Ratio < 95%”
• OS Metrics from SIGAR library
Events and Alerts
• MySQL/ OS Metrics from custom -Graphically seen in console
scripts “Rule Violation Appears on Dashboard”
Notifications
-Requires notification? (SMTP/SNMP)
“Notification Sent”
Expert Advice and Assistance
-Advice on alert/problem and how to fix it
“Increase the query_cache_variable value”
Specific User Action
“Virtual MySQL DBA” - Problem resolved by user intervention

19. MySQL Network Monitoring and
Advisory Service Architecture
MySQL AB Corporate Data Center
Individual
Enterprise Software
Databases
& Updates
Advisor Rules
Server Updates MySQL Enterprise
Web Service Manager
Browser Master-Slave
Enterprise Dashboard Replication
Profiles & Alerts
Knowledge Base
Production Support
MySQL Enterprise
Repository
Multi-vendor
Support, IP
Protection
Cluster

20. MySql Backup
• Dump or Raw Backup
• On-line or Off-line
• Table Types and Consistency
• Storage Requirements
• Replication

21. Dump or Raw Backup
• Dumps
– Can be performed remotely
– Take a lot of space
– CPU intensive and slower
– Are plain SQL, so easy to use later
– Can do selective restore
– Works for any table type
• Raw Backups
– Faster, there is no translation
– Restore “instantly”
– Works for ISAM/MyISAM only

22. On-line or Off-line
• On-line
– May Impacts running applications (can’t write)
– Must deal with data consistency issues
– Slower than off-line because of I/O contention
• Off-line
– Very simple
– Speed is not an issue
– Can use normal backup software

23. Table Types & Consistency
• All tables need to flushed to disk
– FLUSH TABLES
• Tables can’t be updated during backup
– LOCK TABLES...
• Related tables treated as a group, lock all
– FLUSH TABLES WITH READ LOCK
• Transactional tables have logs to backup

24. Storage Requirements
• How long can you store backups?
– Consider staggering older backups
– Never know when you’ll want old data
• Raw backups take less space
– You can keep just the .MYI header
– Rebuild indexes later if need be
• All forms of backup can be compressed

25. Replication
• Backup your slave instead of your master
– Master won’t be interrupted
– Slave can probably be shut down
– Or at least you can hold read locks for a long time
and nobody will care
• It may be worth setting up a slave just for
doing backups
• Remember to backup the master.info file

26. Tools and Techniques
• mysqldump
• mysqlhotcopy
• InnoDB on-line backup
• Off-line backups
• Filesystem snapshots
• Roll your own

27. mysqldump
• Comes with MySQL
• Dumps local or remote tables/databases
• Variety of output formats
• Handles locking if you need it
• Works best for small and medium
installations

28. mysqlhotcopy
• Comes with MySQL
– Written by Tim Bunce of Perl DBI fame
• Raw backups of MyISAM tables
• Handles locking
• Regular expression support
– Match certain databases or tables
• Can truncate indexes to save space

29. mysqlsnapshot
• Separate download
– http://jeremy.zawodny.com/mysql/
• Used for setting up replication snapshots
• Or backing up a running server
• Always backs up everything
– Either one big tar file or one per database
• Very quick to restore

30. InnoDB on-line Backup
• Separate download
– http://www.innodb.com/
• Costs extra $$$
• The only good solution for InnoDB backups
without replication
• Only does InnoDB tables

31. Off-line Backups
• Very easy
• Hard to screw up
• Use any backup tool you like
– Just remember to backup all the MySQL stuff
– You wouldn’t want compatibility problems
after an upgrade
• Restores can be quick

32. Filesystem Snapshot
• Most “expensive” solution
• Need hardware/software support
– Veritas, Network Appliance, LVM, EMC, etc.
• Fastest option available
• Uses least disk space
• Doesn’t guard against all failures

33. MySql Auditing
• Various logs that help in auditing:
• General Query Log
• Binary Logs
• Error Logs
• Slow query log

34. MySQL HA Solutions Overview
• Introduction to MySQL
• HA Terms and Concepts
• HA Requirements and Considerations
• HA Solutions from MySQL
• Third-Party HA Solutions for MySQL
• Additional HA Resources plus Q & A

35. What is High Availability?
• High Availability (HA) refers to the availability of resources in a
computer system despite component failures
• Two major categories
> Customized/redundant software and hardware
> Software-based solutions leveraging COTS components
• Continuous Availability
> Non-stop service
> No disruption of service even during a fail over
> Very high standard of availability, often unnecessary

36. What is Replication?
• The duplication of data to more then one location
• The use of additional hardware or software for…
> Availability
> Reliability
> Fault tolerance
> Performance
• Asynchronous vs. Synchronous
> Safeguarding data loss
• MySQL Replication
> Statement vs Row-based Replication
• Block-level Replication (DRBD)

37. What is Replication?
• The duplication of data changes to more than one location
Asynchronous Replication
update
customer
Synchronous Replication
update
customer

38. What is Clustering?
• Clusters are often used for:
> Operating systems
> Databases
> Application servers
> Load balancers
> Web servers
> Just about any “service” that needs to be highly available
• Redundant nodes provide service when components fail
• Eliminate single points of failure
> Multiple network connections
> Redundant data storage (SAN, RAID)
• Shared-Nothing vs. Shared-Disk

39. MySQL High Availability Solutions
• MySQL Replication
• MySQL Replication and Linux Heartbeat
• MySQL, DRBD and Linux Heartbeat
• MySQL Cluster
• MySQL Cluster Carrier Grade Edition
• MySQL w/ Shared-Storage & Clustering Agents
• Other HA Solutions for MySQL

40. MySQL Replication
• Asynchronous Requirements Support
> Statement Availability
> Row-based (5.1) Automated IP Fail Over No
> Mixed (5.1) Automated DB Fail Over No
• Read vs. Write Typical Fail Over Time Varies
> Response times
Resynchronization of Data No
> Load balancing
Geographic Redundancy Yes
> Scale Out
Scalability
• Administration
Built-in Load Balancing Scale-Out
> Backups
Read Intensive Yes
> Upgrades
Write Intensive No
> Testing
Dozens for
• # of Nodes
Geographic Redundancy Reads

41. MySQL Replication Topologies
Master > Slave Master > Slaves
Masters > Slave (Multi-Source) Master > Slave > Slaves
Circular (Multi-Master)
Master < > Master (Multi-Master)

42. Inside MySQL Replication
Writes & Reads
Web/App
Server
Writes
relay
binlog
mysqld
I/O SQL
Thread Thread
index &
binlogs
data
Replication binlog data
mysqld
MySQL Master MySQL Slave

43. MySQL Replication - Scale Out
• Write to one master
• Read from many slaves
• Perfect for read intensive apps
Load Balancer
Writes & Reads Reads Reads
MySQL Replication

44. MySQL, Heartbeat & DRBD Cluster
IP Management
Active Passive
DRBD DRBD
Server Server
Synchronous
Block Replication

45. Managing Failures
IP Management
Passive X Active
DRBD DRBD
Server Server
X
X
X

46. Data Resynchronization after Host Failure
IP Management
Passive Active
DRBD DRBD
Server Server
Resynchronization
Active Server is still able to process transactions during a resynchronization operation

47. DRBD Basics
• When the failed node comes up
> Becomes a secondary node
> Must synchronize its data to the primary
> Synching happens in the background without service
interruption
> Only the parts of the device which have changed since the
failure will be resynchronized.
• Total resynchronization in just several minutes
> Regardless of device size
> Even after the hard crash of an active node
> Resynch rates may be configured per device
• Device sizes currently limited to 4 TB

48. MySQL Cluster
• Requirements Support
In-Memory Database
> Disk Data (5.1) Availability
• Shared-nothing storage Automated IP Fail Over No
• Synchronous replication Automated DB Fail Over Yes
• Fast-automatic data fail over Typical Fail Over Time < 3 Secs
• High performance
Resynchronization of Data Yes
> NDB API
Yes
• No special hardware Geographic Redundancy
(w/ Replication)
• Best for primary key lookups Scalability
• Transactional
Load Balancing Scale-Out
• Online backup
Read Intensive Yes (Look ups)
Write Intensive Yes
5.0: 64
# of Nodes
5.1: 255

49. MySQL Cluster Carrier Grade Edition
• An advanced version of MySQL Cluster for the telecom industry
• Ideally suited for the telecom industry's top carriers, operators
and equipment manufactures
• Used by industry leaders for real-time, scalable data
management

50. MySQL Cluster Carrier Grade Edition
Characteristic Benefits
• Separate development tree from MySQL Server
Stability • Permits prioritized bug fixes & enhancements for customers
• Developed & tested with major TEMs
Quality & Reliability • Stricter patch and bug acceptance
• Backup and restore status/monitoring
• Support for larger clusters, up to 255 Nodes
• Optimizations to communication protocol enabling API to Data Node over WAN
• Circular replication support
Scalability • Batched slave replication
• Support for user-defined partitioning
• Multiple connections to NDB from a single MySQL Server
• Optimizations to communication protocol enabling API to Data Node over WAN
• More efficient Client/Server data handling and transfers
Performance • Other optimizations (Memory, BLOBs, Disk-based data, etc.)
• Micro Global Checkpoints
• Setting threads to real-time priority
• Online Add Column
• Support for mixed (x86, SPARC, PowerPC) environments
Adaptability • Time-stamp based conflict detection/resolution support in replication
• Locking threads to a CPU

51. MySQL Cluster System Requirements
System Component Requirement
Hosts Maximum of 255 total nodes (48 Data Nodes)
COTS – Advanced TCA
Hardware
32 & 64-bit x86, PowerPC & SPARC
Memory Varies on size of database, # of hosts, # of replicas
Shared-Nothing - Memory & Disk Data
Storage
SCSI or RAID for I/O performance
100 Mbps or better
Network
(Gigabit recommended, SCI supported)
Operating System Linux (Red Hat, SuSE), Solaris, HP-UX, Mac OSX, others…

52. Cluster Components
• Standard SQL Interface
SQL Node • Scale-out for Performance
(MySQL)
• Enables Replication
• High Performance
NDB API • C++ API
(Application)
• Developer’s Guide
• Data Storage (Memory/Disk)
• Automatic & User-Defined Partitioning
Data Node
(NDB Storage Engine) • Local & Global Checkpoints
• Scale-out for Capacity & Redundancy
• Configuration
Management
• Arbitration
Node
• Use Two for Redundancy

53. 2 Data Node MySQL Cluster
Clients
Application/SQL Nodes
(NDB API and/or MySQL Server)
Management
Nodes
Data Nodes
NDB Storage Engine

54. 4 Data Node MySQL Cluster
Clients
Application/SQL Nodes
(NDB API and/or MySQL Server)
Management
Nodes
Data Nodes
NDB Storage Engine

55. Cluster uses Synchronous Replication
Update:
UPDATE author Two-phase
SET country = ‘Cuba’ commit protocol
WHERE authid = 2;
Node Group
authid (PK) fname lname country authid (PK) fname lname country
1 Albert Camus France 1 Albert Camus France
2 Ernest Hemingway Cuba 2 Ernest Hemingway Cuba
3 Johan Goethe Germany 3 Johan Goethe Germany
4 Junichiro Tanizaki Japan 4 Junichiro Tanizaki Japan

56. Geographic Redundancy
• As of 5.1, support for asynchronous replication between
Clusters
• Enables a higher degree of availability and continuity
Asynchronous MySQL Replication
NDB API
NDB API
NDB Storage Engine
NDB Storage Engine
Cluster 1 Cluster 2

57. Distribution & Partitioning: 4 Data Nodes
Four Data Nodes (Physical)
authid (PK) fname lname country * Half of database on each node
1 Albert Camus France
2 Ernest Hemingway USA
Two Replicas (Logical)
* Two copies of each partition
3 Johan Goethe Germany
4 Junichiro Tanizaki Japan
Two Node Groups
* Two sets of identical Data Nodes
Node Group 1 Node Group 2
authid (PK) fname lname country authid (PK) fname lname country
1 Albert Camus France 2 Ernest Hemingway USA
3 Johan Goethe Germany 4 Junichiro Tanizaki Japan

58. Distribution & Partitioning: 4 Data Nodes
Query:
SELECT fname, lname
FROM author The query can only be satisfied by a
WHERE authid = 2;
Data Node belonging to Node Group 2
Result: because the data is not stored within
Node Group 1
Ernest Hemingway
Node Group 1 Node Group 2
authid (PK) fname lname country authid (PK) fname lname country
1 Albert Camus France 2 Ernest Hemingway USA
3 Johan Goethe Germany 4 Junichiro Tanizaki Japan

59. Distribution & Partitioning: 4 Data Nodes
Query:
SELECT fname, lname
FROM author
In a four node cluster, even if we
WHERE authid BETWEEN 1 AND 3;
loose a Data Node in each Node
Result: Group, we can still satisfy any
query
Albert Camus
Ernest Hemingway
Johan Goethe
Node Group 1 Node Group 2
authid (PK) fname
X
lname country
X
authid (PK) fname lname country
1 Albert Camus France 2 Ernest Hemingway USA
3 Johan Goethe Germany 4 Junichiro Tanizaki Japan

60. Data Node Failure & Recovery
• Server 3 failure is detected
• Server 4 handles Server 3 requests
1 2 3 4
Node Group 1 Node Group 2

61. Data Node Failure & Recovery
• Server 3 begins copying data from Server 4
• Complete or just changes since failure
1 2 3 4
Node Group 1 Node Group 2

62. Data Node Failure & Recovery
• Once data is fully restored, Server 3 rejoins the cluster
1 2 3 4
Node Group 1 Node Group 2

63. Network Partitioning – Avoiding “Split Brain”
• By default, the Management Node acts as arbitrator
• In the event of a network failure, arbitration determines which set of
nodes should continue while shutting down the rest
NDB API
Application
Nodes
(NDB API and/or MySQL Server)
Management
Nodes
Data Nodes
X

64. MySQL MySQL Replication MySQL, Heartbeat
Requirements MySQL Cluster
Replication + Heartbeat + DRBD
Availability
Automated IP Fail Over No Yes Yes No
Automated DB Fail Over No No Yes Yes
Typical Fail Over Time Varies Varies < 30 secs < 3 secs
Auto Resynch of Data No No Yes Yes
Geographic Redundancy Yes Yes MySQL Replication MySQL Replication
Scalability
MySQL
Built-in Load Balancing MySQL Replication MySQL Replication Yes
Replication
Read Intensive Yes Yes MySQL Replication Yes
Write Intensive No No If configured correctly Yes
# of Nodes per Cluster Master/Slave(s) Master/Slave(s) Active/Passive 255
# of Slaves Dozens for Reads Dozens for Reads Dozens for Reads Dozens for Reads

65. MySQL Migration Toolkit
• Wizard Interface. Supports...
• Oracle
• Microsoft SQL Server
• Microsoft Access
• Sybase
• MySQL

66. Questions...

67. Thank you for your Time
and Attention!
For more information visit http://www.osscube.com
Or drop-in an email to mysql@osscube.com