Observing Your App and Everything Else Using DTrace

Observing Your App and
Everything Else It Runs On Using
DTrace
session code S308357

Angelo Rajadurai
Sr. Technology Evangelist – Sun Microsystems Inc.

Sunday, May 31, 2009

goal

Learn how to Dynamically observe your
application and its environment.

2009 CommunityOne Conference: EAST | developers.sun.com/events/communityone


your app...

Application Logic
ruby
Frameworks
python
Presentation php

C++
C
JSP/Javascript/Ajax/JavaFX
perl
Container Glassfish/J2EE shell
java
Database Apps run Native
in VM Apps
MySQL/Oracle/Postgres

Operating System Solaris/Linux/OS X/Windows
2009 CommunityOne Conference: EAST | developers.sun.com/events/communityone 3


on the menu today

Our Setup
Drupal 6.12

Apache 2.2
PHP - Zend 5.2

MySQL 5.1.30
Amp Stack

OpenSolaris 2009.06
4

2009 CommunityOne Conference: EAST | developers.sun.com/events/communityone


one small easy step

the AMP stack
To build, install, integrate, performance tune and deploy
the amp stack on opensolaris...

# pkg install amp-dev



one slide on Drupal



questions questions questions

How much time am I
spending in Apache/
How much of my load is MySQL/PHP?
from anonymous users

What are the Drupal
modules that take up
the most time?

What type of database
load is caused by
anonymous users What are the most
expensive hooks in a
given Drupal module?



the big game hunt

“M
ay
m in” Da
be ta
d
r: e a th bas
pe as
lo b
e
w i m Ad
e
ve ata ll idd mi
De d kn le n:
o w wa
e
th ” re
k gu
as ys
“I’d

“I :
do S
n’t ysa e rt in”
xp adm
De kno dmi E
e ys
ve w! n ar e S
lop A
e r sk d lew th
” th id sk
e M a
s
et
“L


the vicious cycle

Restart
Stop App Add Code Compile
w/ ﬂags

Prebuilt Instruments Custom Instruments

Instrument Validate

Run in
production

Process Collect info



oh please

Please let me
observe a live
system.



dynamic tracing

Application Logic
Frameworks c
Presentation c++
12::SELECT id, name, symbol, conversion_rate FROM currencies WHERE
JSP/Javascript/Ajax
status = 'Active' and deleted = 0
12::SELECT category, name, value FROM config
ph

pytho
12::SELECT id FROM outbound_email WHERE type = 'system'

rub
Container Glassﬁsh/Tomcat
perl
Database shell
Native
Operating System Solaris/Linux/OS X/


dynamic tracing

Application Logic
-> contentAreaClick
-> getPreventDefault

c
<- getPreventDefault

Frameworks
<- contentAreaClick

Presentation c++
JSP/Javascript/Ajax
ph

pytho
rub
Container Glassﬁsh/Tomcat
perl
Database shell
Native
Operating System Solaris/Linux/OS X/


You can do it with...

DTrace



poll

Have you heard about it

Have you used it

Have you written d-scripts



The D - Language

Basics
of
Dynamic Instrumentation



instrumentation format

d-script
probe
provider : module : probe : name where

/ predicate / when
{

actions do what

}



command line version

dtrace command line

# dtrace -n probe’/predicate/{action}’



demo



aggregation

dtrace -P syscall
=> mmap mmap
5 <= mmap mmap
5 => munmap munmap
5 <= munmap munmap
5 => setcontext setcontext
5 <= setcontext setcontext
5 => getrlimit getrlimit
5 <= getrlimit getrlimit
5 => getpid getpid
5 <= getpid getpid
5 => setcontext setcontext
5 <= setcontext setcontext
5 => sysi86 sysi86
5 <= sysi86 sysi86
5 => brk brk
5 <= brk brk
5 => brk brk
5 <= brk brk
5 => xstat xstat
5 <= xstat xstat
5 => resolvepath resolvepath
5 <= resolvepath resolvepath
5 => open open
5 <= open
5 => mmap
open
mmap rexit 1
5 <= mmap
5 => mmap
mmap
mmap fcntl 2
fsat 2
5 <= mmap mmap
5 => mmap mmap
5 <= mmap mmap
5 => mmap
5 <= mmap
mmap
mmap getpid 2
5 => munmap
5 <= munmap
munmap
munmap getrlimit 2
sysi86 2
5 => memcntl memcntl
5 <= memcntl memcntl
5 => close close
5 <= close
5 => xstat
close
xstat write 2
fstat64 4
5 <= xstat xstat
5 => resolvepath resolvepath
5 <= resolvepath resolvepath
5 => open
5 <= open
open
open getdents64 4
5 => mmap
5 <= mmap
mmap
mmap memcntl 4
open 4
5 => mmap mmap
5 <= mmap mmap
5 => mmap mmap
5 <= mmap
5 => mmap
mmap
mmap resolvepath 4
5 <= mmap
5 => mmap
5 <= mmap
mmap
mmap
mmap
setcontext 4
5 => munmap
5 <= munmap
munmap
munmap xstat 4
5 => memcntl
5 <= memcntl
5 => close
memcntl
memcntl
close
ioctl 6
5 <= close
5 => munmap
close
munmap brk 8
5 <= munmap
5 => ioctl ioctl
munmap
munmap 8
close 10
5 <= ioctl ioctl
5 => ioctl ioctl
5 <= ioctl ioctl
5 => brk
5 <= brk
5 => brk
brk
brk
brk
mmap 20
5 <= brk brk
5 => fsat fsat
5 <= fsat fsat
5 => fcntl fcntl
5 <= fcntl fcntl
5 => fstat64 fstat64
5 <= fstat64 fstat64
5 => getdents64 getdents64
5 <= getdents64 getdents64
5 => getdents64 getdents64

dtrace -P syscall’{@=count(probefunc)}’
5 <= getdents64 getdents64
5 => close close
5 <= close close
5 => ioctl ioctl
5 <= ioctl ioctl
5 => fstat64 fstat64
5 <= fstat64 fstat64
5 => write write
5 <= write write
5 => close close
5 <= close close
5 => close close
5 <= close close
5 => rexit rexit



dtrace aggregates - demo



the pid provider

• Dynamically create probes in
provider: pid$<processId> live running applications.
module: name of library
probe: function name
• No extra work needed from
name: entry application developer
return
offset
• Available without any compile
or runtime flags
• Can be created on optimized
Pr

binaries as well
Pi id
ov

• Probes created at function
d er

boundaries and offset



demo



probing MySQL using Pid

provider: pid$<mysql-pid>
module: not-used
probe:*dispatch_command* #!/usr/sbin/dtrace -qs
name: entry
arg2 - SQL command pid$1::*dispatch_command*:entry
{
printf("%sn",copyinstr(arg2,arg3));
}
M ue
Q
yS ri
Q es
L



dtrace and mysql - demo



php provider
New and improved in OpenSolaris 2009.06
compile-file-entry/-return
function-entry/-return
provider: php$<pid>
module: not-used execution-entry/-return
probe: not-used exception-thrown/-catched
name: <see detail ->
error
request-startup/-shutdown
Ap ov
Pr

Older releases
ac id
he er

function-entry/-return


dtrace and php - demo



Apache provider

received-request
provider: apache$<pid>
module: not-used log-request
probe: not-used
name: <see detail -> create-child
accept-connection
check-user-credentials
check-access
Ap ov
Pr

check-authorization
ac id
he er



gotcha



dtrace and the webstack



Observing the entire stack
More details on the scripts we saw earlier...

what are the hot modules in Drupal?

what are the hot hooks in Drupal?

how much of my Drupal load is from anonymous users

how much time am I spending in Apache/MySQL/PHP?

what are the SQL statements that are executed by anon
users


Give me more!
DTrace landing page:
http://www.opensolaris.org/os/community/dtrace

DTrace Toolkit:
In OpenSolaris - /opt/DTT
Solaris download -
http://www.opensolaris.org/os/community/dtrace/dtracetoolkit/

DTrace Bigadmin page:
http://www.sun.com/bigadmin/content/dtrace/

Lots and lots of info on the web - Google “dtrace”



Social
My Blog:
http://blogs.sun.com/angelo

Twitter:
rajadurai

DTrace hands on lab
http://developers.sun.com/learning/javaoneonline/j1lab.jsp?
lab=LAB-9400&yr=2008&track=1



Observing Your App and
Everything Else It Runs On Using
DTrace
session code S308357

Angelo Rajadurai
angelo.rajadurai@sun.com


DTrace and JavaScript



Observing javascript using the javascript provider

Enter script details



implementation



Observing Your App and Everything Else Using DTrace

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (11)

Semelhante a Observing Your App and Everything Else Using DTrace

Semelhante a Observing Your App and Everything Else Using DTrace (20)

Último

Último (20)

Observing Your App and Everything Else Using DTrace