This document summarizes MySQL's monitoring mechanisms and how they have evolved over time. It discusses tools like SHOW statements, INFORMATION_SCHEMA, slow/general query logs, and EXPLAIN that provided limited visibility in past versions. MySQL 5.5 introduced the Performance Schema framework for detailed instrumentation. Subsequent versions have expanded instrumentation to provide more developer-focused statistics on statements, stages, I/O, locks and more. New INFORMATION_SCHEMA tables in 5.6 provide additional InnoDB statistics on data dictionary, buffer pool, transactions and compression. The optimizer trace exposes query transformations. Enhanced EXPLAIN now supports more statement types and future improvements will provide a structured EXPLAIN output.

2. MySQL Monitoring Mechanisms: How the different pieces fit together Mark Leith (@MarkLeith)

3. Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

5. The New MySQL Monitoring Vision

6. Performance Schema

7. New InnoDB Instrumentation

8. Optimizer Trace

9. Enhanced EXPLAIN

10. Other Goodies

11. Summary

12. A long time ago in a galaxy far, far away.... MySQL Episode IV.I The SHOW statement continues to rule supreme... USE THE FORCE LUKE!!

14. INFORMATION_SCHEMA for (mostly) object metadata

15. Slow / General query logs for statement history

16. EXPLAIN / SHOW PROFILE for statement profiling

19. Can not filter, or re-order columns

20. Can not use dynamically (prepared statements etc.)

26. CSV based log tables in 5.1+ are not ideal

30. EXPLAIN doesn't support all statement types

31. SHOW PROFILE often isn't fine grained enough

32. MySQL Episode V.V A NEW HOPE

36. Compression statistics

38. Expose even more detailed InnoDB statistics

39. Take the guess work out of optimizer workings

40. Increase flexibility of instrumentation configuration

41. Provide more structured data for richer tooling

43. INFORMATION_SCHEMA – Standard Object Metadata

44. INFORMATION_SCHEMA – Plugin Specific Extensions

45. PERFORMANCE_SCHEMA – Consolidated Statistics

46. Performance Schema

48. Similar in goals to the “ Oracle Wait Interface ”

49. Tracks Events by time/latency (to picosecond precision)

50. Entirely non-locking, and in fixed memory

52. Not persistent

57. Read/Write locks - wait/synch/rwlock/

59. Performance Schema in 5.5 THREAD_ID: 17 EVENT_ID: 57265474 EVENT_NAME: wait/synch/mutex/innodb/kernel_mutex SOURCE: lock0lock.c:5489 TIMER_START: 1716417836794180 TIMER_END: 1716417836828110 TIMER_WAIT: 33930 SPINS: NULL OBJECT_SCHEMA: NULL OBJECT_NAME: NULL OBJECT_TYPE: NULL OBJECT_INSTANCE_BEGIN: 4307564056 NESTING_EVENT_ID: NULL OPERATION: lock NUMBER_OF_BYTES: NULL FLAGS: 0 Mutex Event ~33 nanosecs To acquire lock

60. Performance Schema in 5.5 SELECT event_name, ROUND(SUM(sum_timer_wait) / 1000000000000, 2) total_sec, ROUND(MIN(min_timer_wait) / 1000000, 2) min_usec, ROUND((SUM(sum_timer_wait) / SUM(COUNT_STAR))/1000000, 2) avg_usec, ROUND(MAX(max_timer_wait) / 1000000, 2) max_usec FROM performance_schema.events_waits_summary_global_by_event_name WHERE sum_timer_wait > 0 GROUP BY event_name ORDER BY total_sec DESC limit 10; Top waits by latency

61. Performance Schema in 5.5 THREAD_ID: 6 EVENT_ID: 1036 EVENT_NAME: wait/io/file/innodb/innodb_data_file SOURCE: os0file.c:4891 TIMER_START: 338825682207740 TIMER_END: 338825703463000 TIMER_WAIT: 21255260 SPINS: NULL OBJECT_SCHEMA: NULL OBJECT_NAME: /Users/mark/sandboxes/msb_5_5_16/data/ibdata1 OBJECT_TYPE: FILE OBJECT_INSTANCE_BEGIN: 4683519488 NESTING_EVENT_ID: NULL OPERATION: write NUMBER_OF_BYTES: 32768 FLAGS: 0 File IO Event ~21 microsecs To write 32K On this file

62. Performance Schema in 5.5 SELECT substring_index(file_name, '/', -1) AS file, count_read, count_write, ROUND(sum_number_of_bytes_read / (1024*1024)) read_mb, ROUND(sum_number_of_bytes_write / (1024*1024)) write_mb, ROUND((sum_number_of_bytes_read + sum_number_of_bytes_write) / (1024*1024*1024)) total_gb FROM performance_schema.file_summary_by_instance ORDER BY total_gb DESC LIMIT 10; Top files by IO consumption

64. Already aided finding bottlenecks

66. Needed more control on who/what is traced

67. Needed a little tuning for high load systems

69. With the ability to drill down between each level

71. Table IO - at the storage engine handler layer

72. Table Locks - at the storage engine handler layer

73. .. as well as file IO , mutexes , conditions , rwlocks

74. Performance Schema Statement Statistics

75. Performance Schema Stages The “stage” (state) Within statement execution When it started, ended, and How long it took The statement ID it was related to

77. Currently uses the “Adjacency List” model

78. EVENT_ID (child) -> NESTING_EVENT_ID (parent)

79. Investigating the “Nested Set” model

80. Performance Schema in 5.6 – Event Nesting

81. Performance Schema in 5.6 – Graph Events http://www.markleith.co.uk/?p=471 The Statement The Stages The individual wait events by type

82. Performance Schema in 5.6 – Graph Events

83. Performance Schema in 5.6 – Summaries Summarize (GROUP BY) each class of event by user, host, account ( [email_address] ), thread, and globally New Object, Network IO Table IO and Table Lock Wait summaries

84. Performance Schema Table IO Summary The table and it's high level aggregated stats A breakdown By table IO type

85. “ Not everything that can be counted counts, and not everything that counts can be counted.” Albert Einstein Yet 60 nanoseconds of time could mean SO MUCH!

87. Whether, and how, we time events can be important

89. How selectively you want to monitor it

90. Performance Schema Overhead http://marcalff.blogspot.com/2011/06/performance-schema-overhead-tuning.html June 2011 Lots of tuning done since this. New benchmarks should be out soon

91. Performance Schema Setup

93. User lock latency – i.e. SELECT GET_LOCK();

94. A normalized statement level digest

95. Transactions level statistics (tied to global transaction ID)

96. Memory allocations

97. Distribution statistics (histograms)

98. Background thread “stages”, server cache statistics

99. ...and...and...and... - We're listening, tell us your needs

100. New InnoDB Instrumentation

102. 3 Buffer Pool related

104. 4 Compression related

105. InnoDB System Tables http://dev.mysql.com/doc/refman/5.6/en/innodb-sys-tablestats-table.html View InnoDB's internal Data Dictionary and table stats

106. INNODB_SYS_TABLES[TATS] SELECT innodb_sys_tables.name, CASE flag >> 5 WHEN 0 THEN 'Antelope' WHEN 1 THEN 'Barracuda' WHEN 2 THEN 'Cheetah' END AS file_format, @page_flags := (flag >> 1) & 15 AS page_flags, @page_size := IF(@page_flags = 0,16384, 512 << @page_flags) AS page_size, ROUND(( @page_size * clust_index_size ) /(1024*1024)) AS pk_mb, ROUND(( @page_size * other_index_size ) /(1024*1024)) AS secidx_mb FROM innodb_sys_tables JOIN innodb_sys_tablestats USING (table_id) ORDER BY pk_mb DESC;

107. INNODB_BUFFER_PAGE Tables

108. INNODB_BUFFER_PAGE - Summarized http://dev.mysql.com/doc/refman/5.6/en/innodb-buffer-page-table.html SELECT page_type, page_state, table_name, index_name, SUM(data_size) / (1024*1024) AS used_mb, SUM( IF(compressed_size = 0, 16384, compressed_size) ) / (1024*1024) AS total_mb FROM innodb_buffer_page GROUP BY page_type, page_state, table_name, index_name HAVING total_mb > 0 ORDER BY total_mb DESC;

110. 199 metrics currently collected ( suggest just one more! )

112. Track when / how long each has been enabled

113. Provide extra statistics inline per metric

114. Provide extra metadata inline per metric

115. INNODB_METRICS – Metric Details http://dev.mysql.com/doc/refman/5.6/en/innodb-metrics-table.html The variable Stats since start Stats since “reset” How long enabled Stat Metadata

116. INNODB_METRICS – Configuration Enable named metric Disable named metric Enable metric “module” Reset metric “module”

117. INNODB_METRICS – Looking at Counters http://dev.mysql.com/doc/refman/5.6/en/innodb-metrics-table.html SELECT concat(subsystem,'/', name) AS event, count, ROUND(count / time_elapsed) AS avg_per_second FROM information_schema.innodb_metrics WHERE status = 'enabled' AND type LIKE '%counter%' OR type = 'set_owner' ORDER BY avg_per_second DESC;

118. Optimizer Trace

120. Optimizer Trace tells you why it will execute that way

121. New INFORMATION_SCHEMA.OPTIMIZER_TRACE

122. Contains trace of optimizer decisions, in JSON format

123. Enabled per thread, on demand

124. Targeted for advanced users, or in use with bug reporting

125. Optimizer Trace Enable Tracing Run Statement Select the trace

126. Optimizer Trace Expose query transformations

127. Optimizer Trace Considered plans With costs Estimations Of rows And costs Whether Indexes are Usable or not

128. Enhanced EXPLAIN

132. A properly structured EXPLAIN plan (again, JSON)

133. Sorry – no examples yet, it's very early in implementation

134. Other Goodies

136. Persisted in $datadir/auto.cnf

138. Q&A