RubyConfPH 2014

1. High Performance Rails with MySQL
Jervin Real, March 2014

2. I am…
• Consultant, Percona
• @dotmanila
• http://dotmanila.com/blog/
• http://www.mysqlperformanceblog.com/

3. Rails Fu Mastah!
http://walksalong.files.wordpress.com/2007/05/bruce_on_rails.jpg

4. Beginner
http://www.devonring.ca/img/ouch.png

5. Customer Problems

6. Web Apps Performance
• Powerful servers
• CPUs, higher clock speeds
• Lots of memory
• Fast storage
• Scale in the cloud
• Agile development techniques

7. Why Not?
• Premature scaling is expensive
• Cost inefficient
• Agile means less effective measurement to compensate
for fast deployments

8. Squeeze the Software
• Exhaust application optimizations first
• You cannot optimize what you can’t measure
• Cacti, NewRelic, Scout
• NewRelic RPM Developer Mode, Rails Footnotes (2, 3,
4!), Google PerfTools for Ruby

9. • Dumb schemas and queries (somewhat)
Characteristics of Rails
:primary_key, :string, :text, :integer, :float, :decimal,
:datetime, :timestamp, :time, :date, :binary, :boolean

10. • Dumb schemas
• Likes to use SHOW FIELDS
Characteristics of Rails

11. • Dumb schemas
• Likes to use SHOW FIELDS
• N+1 queries problem
• Well documented and discouraged for use
Characteristics of Rails

12. • Dumb schemas
• Likes to use SHOW FIELDS
• N+1 queries problem
• Well documented and discouraged for use
• Does SELECT FOR UPDATE
• NOOP transactions still wrapped in BEGIN/COMMIT
Characteristics of Rails

13. • FK relationships - logical - GOOD
Rails - Good

14. • FK relationships - logical - GOOD
• Knows how to use PRIMARY KEYs — VERY GOOD!
Rails - Good

15. • FK relationships - logical - GOOD
• Knows how to use PRIMARY KEYs — VERY GOOD!
• Knows NOOP changes - GOOD
Rails - Good

16. • FK relationships - logical - GOOD
• Knows how to use PRIMARY KEYs — VERY GOOD!
• Knows NOOP changes - GOOD
• Database agnostic - GOOD
Rails - Good

17. MySQL by Default
• Assumes you have less powerful hardware
• ironically on 5.5, assumes you have very powerful
CPUs - innodb_thread_concurrency = 0

18. MySQL by Default
• Assumes you have less powerful hardware
• ironically on 5.5, assumes you have very powerful
CPUs - innodb_thread_concurrency = 0
• Not optimized for faster storage

19. MySQL by Default
• Assumes you have less powerful hardware
• ironically on 5.5, assumes you have very powerful
CPUs - innodb_thread_concurrency = 0
• Not optimized for faster storage
• Still have bad configuration assumptions
• Query cache
• MyISAM < 5.5.5

20. MySQL by Default
• Assumes you have less powerful hardware
• ironically on 5.5, assumes you have very powerful
CPUs - innodb_thread_concurrency = 0
• Not optimized for faster storage
• Still have bad configuration assumptions
• Query cache
• MyISAM < 5.5.5
• Still haunted by mutexes

21. What to Optimize - MySQL
• Disable Query Cache

22. What to Optimize - MySQL
• Disable Query Cache
• query_cache_size = 0

23. What to Optimize - MySQL
• Disable Query Cache
• query_cache_size = 0
• query_cache_type = 0

24. What to Optimize - MySQL
• Disable Query Cache
• skip_name_resolve

25. What to Optimize - MySQL
• Disable Query Cache
• skip_name_resolve
• Use >= 5.5

26. What to Optimize - MySQL
• Disable Query Cache
• skip_name_resolve
• Use >= 5.5
• Use Indexes, EXPLAIN should be your friend!

27. What to Optimize - MySQL
• Disable Query Cache
• skip_name_resolve
• Use >= 5.5
• Use Indexes, EXPLAIN should be your friend!
• 5.6 does Subquery Optimizations

28. What to Optimize - MySQL
• Slow queries - search and destroy
• long_query_time = 0
• log_slow_verbosity - Percona Server
• pt-query-digest/Percona Cloud Tools

29. What to Optimize - MySQL
• Use InnoDB
innodb_buffer_pool_size # keep hot data in memory
innodb_log_file_size # allow more IO buffer
innodb_flush_method = O_DIRECT # skip OS cache
innodb_[read|write]_io_threads > 4
innodb_io_capacity
innodb_adaptive_flushing_method

30. What to Optimize - Rails
• counter_cache
• Good for InnoDB if you often SELECT COUNT(*)
• Indexes for find_by, where and family
@posts = Post.where(title: params[:keyword])
!
mysql> EXPLAIN SELECT `posts`.* FROM `posts` WHERE `posts`.`title` = 'LongTitles' G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: posts
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 2
Extra: Using where
1 row in set (0.00 sec)

31. What to Optimize - Rails
• dependent: :destroy
24 Query BEGIN
24 Query SELECT `comments`.* FROM `comments` WHERE `comments`.`post_id` = 1
24 Query DELETE FROM `comments` WHERE `comments`.`id` = 2
24 Query DELETE FROM `posts` WHERE `posts`.`id` = 1
24 Query COMMIT
24 Query BEGIN
24 Query DELETE FROM `comments` WHERE `comments`.`post_id` = 4
24 Query DELETE FROM `posts` WHERE `posts`.`id` = 4
24 Query COMMIT
BAD
Use dependent: :delete_all

32. What to Optimize - Rails
• Cache SHOW FIELDS output - patches for now
• Disable innodb_stats_on_metadata

33. What to Optimize - Rails
• Cache SHOW FIELDS output - patches for now
• Disable innodb_stats_on_metadata
• Pull only the columns you need - especially excluding
BLOBS
@posts = Post.select(“title”)

34. What to Optimize - Rails
• Avoid pessimistic locks when possible - SELECT … FOR
UPDATE - UPDATE directly and return affected rows

35. What to Optimize - Rails
• Avoid pessimistic locks when possible - SELECT … FOR
UPDATE - UPDATE directly and return affected rows
• Avoid N+1 queries - use JOIN or includes

36. What to Optimize - Rails
@posts = Post.limit(2)
!
@posts.each do |post|
puts post.authors.name
end
24 SELECT `posts`.* FROM `posts` LIMIT 2
24 Query SELECT `authors`.* FROM `authors` WHERE
`authors`.`id` = 1 ORDER BY `authors`.`id` ASC LIMIT 1
24 Query SELECT `authors`.* FROM `authors` WHERE
`authors`.`id` = 2 ORDER BY `authors`.`id` ASC LIMIT 1
With this:
You get this:

37. What to Optimize - Rails
@posts = Post.includes(:authors).limit(2)
24 Query SELECT `posts`.* FROM `posts` LIMIT 2
24 Query SELECT `authors`.* FROM `authors`
WHERE `authors`.`id` IN (1, 2)
With this:
You get this:

38. What to Optimize - Rails
@posts = Post.joins(:authors).limit(2);
24 Query SELECT `posts`.* FROM `posts` INNER
JOIN `authors` ON `authors`.`id` =
`posts`.`authors_id` LIMIT 2
With this:
You get this:

39. What to Optimize - Rails
• Avoid pessimistic locks when possible - SELECT … FOR
UPDATE - UPDATE directly and return affected rows
• Avoid N+1 queries - use JOIN or includes
• Compress MySQL requests, especially transactions!

40. What to Optimize - Rails
• Avoid pessimistic locks when possible - SELECT … FOR
UPDATE - UPDATE directly and return affected rows
• Avoid N+1 queries - use JOIN or includes
• Compress MySQL requests, especially transactions!
• Learn and use SQL - find_by_sql

41. What to Optimize - Rails
• Avoid pessimistic locks when possible - SELECT … FOR
UPDATE - UPDATE directly and return affected rows
• Avoid N+1 queries - use JOIN or includes
• Compress MySQL requests, especially transactions!
• Learn and use SQL - find_by_sql
• Don’t accept Model defaults

42. Further Thoughts
• GDB, Strace, OProfile
• Smart aggregation, use summary tables when possible
• Rails > Caching > MySQL
• Avoid:
config.action_controller.session_store = :active_record_store

43. http://www.amyvernon.net/wp-content/uploads/2014/01/upward-graph-striving.png

44. Thank you!

45. … and
• We’re hiring!
• http://www.percona.com/about-us/careers/open-
positions