1. Profiling and monitoring
ruby/rails
JÁN SUCHAL
@JSUCHAL

2. Optimalization
 “If you can’t measure it, you can’t improve it.” – Lord Kelvin
 Environment
 development vs. production (hw, sw, load)
 data (synthetic vs. real)
 Bottlenecks
 10sec * 1 run vs. 0.5s * 100 runs
 run time * runs vs. development time
 Microbenchmarks
 waste of time vs. 20 * 1% = 20%
 “There are three kinds of lies: lies, damned lies, and statistics.” –
Benjamin Disraeli
 single run vs. multiple runs
 average vs. standard deviance, percentiles
 cold vs. hot cache

3. Profiler - Example
# profiler/dates.rb
require 'date'
def create_days_after(date_str, n)
after = Date.strptime(date_str) + n
after.strftime("%Y-%m-%d")
end
1000.times do
create_days_after("1982-10-27", 5)
end

4. Ruby Profiler
 $ gem install ruby-prof
 $ ruby-prof dates.rb
 How long does each method take?
 $ ruby-prof dates.rb –m 3
 Just methods above 3% time
Thread ID: 7749480
Total: 0.653076
%self total self wait child calls name
12.97 0.19 0.08 0.00 0.18 2000 String#scan
11.57 0.08 0.08 0.00 0.00 205000 String#===
6.87 0.18 0.04 0.00 0.14 1000 String#gsub
6.28 0.05 0.04 0.00 0.01 14000 Hash#values_at
4.08 0.03 0.03 0.00 0.00 80470 Hash#default
3.21 0.03 0.02 0.00 0.01 3000 Date#emit

5. Ruby Profiler
Thread ID: 7749480
Total: 0.653076
%self total self wait child calls name
12.97 0.19 0.08 0.00 0.18 2000 String#scan
11.57 0.08 0.08 0.00 0.00 205000 String#===
6.87 0.18 0.04 0.00 0.14 1000 String#gsub
6.28 0.05 0.04 0.00 0.01 14000 Hash#values_at
4.08 0.03 0.03 0.00 0.00 80470 Hash#default
3.21 0.03 0.02 0.00 0.01 3000 Date#emit
 total – time in method and children calls
 self – time in method call
 wait – wait time
 child – time in child calls
 call – number of times method invoked

6. Ruby Profiler
 $ ruby-prof dates.rb -m 3 -p graph
 Which method calls what and how many times?
Thread ID: 15523700
Total Time: 0.675587048
%total %self total self wait child calls Name
0.20 0.10 0.00 0.20 2000/2000 <Class::Date>#_strptime_i
30.28% 14.70% 0.20 0.10 0.00 0.20 2000 String#scan
0.04 0.04 0.00 0.00 110000/205000 String#===
0.02 0.01 0.00 0.01 3000/5000 Date::Format::Bag#method_...
0.02 0.00 0.00 0.01 2000/3005 Class#new
0.01 0.01 0.00 0.00 5000/5000 String#sub!
0.01 0.00 0.00 0.01 2000/2000 Range#===
0.00 0.00 0.00 0.00 3000/3000 String#to_i
0.00 0.00 0.00 0.00 2000/2000 <Class::Regexp>#quote
0.00 0.00 0.00 0.00 2000/2000 Regexp#===
0.00 0.00 0.00 0.00 1000/1000 <Class::Date>#num_pattern?
0.00 0.00 0.00 0.00 1000/2000 <Class::Date>#_strptime_i

7. Ruby Profiler
Thread ID: 15523700
Total Time: 0.675587048
%total %self total self wait child calls Name
0.20 0.10 0.00 0.20 2000/2000 <Class::Date>#_strptime_i
30.28% 14.70% 0.20 0.10 0.00 0.20 2000 String#scan
0.04 0.04 0.00 0.00 110000/205000 String#===
0.02 0.01 0.00 0.01 3000/5000 Date::Format::Bag#method_..
 three parts
 parent calls
 method
 children calls
 calls – number of calls from method/total number of calls

8. Ruby Profiler
 $ ruby-prof dates.rb -p graph_html -m 3 > graph.html
 $ ruby-prof dates.rb -p graph_html -m 3 –s self > graph.html

9. Profiler - KCacheGrind
 $ ruby-prof dates.rb -p call_tree -m 3 > dates1.grind
 $ kcachegrind dates1.grind

10. benchmark-ips
require 'benchmark/ips'
require 'ostruct'
require 'hashr'
require 'hashugar'
SMALL_HASH = {:a => 1, :b => 2}
Benchmark.ips do |x|
x.report 'OpenStruct create small hash and access once', 'OpenStruct.new(SMALL_HASH).item5'
x.report 'Hashr create small hash and access once', 'Hashr.new(SMALL_HASH).item5'
x.report 'Hashugar create small hash and access once', 'Hashugar.new(SMALL_HASH).item5‘
end
OpenStruct create small hash and access once
43858.0 (±5.5%) i/s - 221820 in 5.074250s (cycle=3697)
Hashr create small hash and access once
67408.9 (±5.0%) i/s - 339780 in 5.053728s (cycle=5663)
Hashugar create small hash and access once
230217.9 (±4.2%) i/s - 1152670 in 5.015705s (cycle=15790)

11. Memory profiling
 Patched ruby
 $ rvm install 1.9.3 --patch railsexpress --name gc
 $ ruby-prof --mode=allocations dates.rb –m 3
%self total self wait child calls name
59.23 6004.00 6004.00 0.00 0.00 1000 <Class::Date>#strptime
9.87 1000.00 1000.00 0.00 0.00 1000 Date#strftime
9.87 9004.00 1000.00 0.00 8004.00 1000 Object#create_days_after
9.87 1000.00 1000.00 0.00 0.00 1000 Date#+
9.87 10004.00 1000.00 0.00 9004.00 1 Integer#times

12. Rails / NewRelic Developer mode
 gem 'newrelic_rpm‘
 http://localhost:3000/newrelic
 gem 'newrelic_rpm', git:
'git://github.com/jsuchal/rpm.git',
branch: 'feature-profile-sorting'

13. NewRelic Developer mode

14. Rails / NewRelic Developer mode

15. NewRelic Developer mode

16. NewRelic Developer mode Profiler

17. Custom Method Tracers
# initializers/elastic_search_traces.rb
require 'new_relic/agent/method_tracer‘
ElasticSearch.class_eval do
include NewRelic::Agent::MethodTracer
add_method_tracer :search, 'Custom/elasticsearch/search'
add_method_tracer :index, 'Custom/elasticsearch/index'
end

18. NewRelic Production Monitoring
 Web Transactions – controller actions drilldown
 Transaction Traces – detailed slow requests
 Slow SQL – slow queries
 Background job monitoring
 Availabality monitoring
 Deployment tracking
 Scalability analysis
 ...
 Server monitoring (load, disks, …)

23. Error tracking
 ExceptionNotifier
 www.airbrake.io

24. Most common performance problems
 “1 + N query problem”
 joins FTW!
 Lack of proper indexing
 Unnecessary ActiveRecord loading
 e.g. count vs. size vs. length
 Default GC parameters
 37signals params
 RUBY_HEAP_MIN_SLOTS=600000 # This is 60(!) times larger than default
 RUBY_GC_MALLOC_LIMIT=59000000 # This is 7 times larger than default
 RUBY_HEAP_FREE_MIN=100000 # This is 24 times larger than default
 Unknown abstraction internals
 e.g. OpenStruct