Oracle continues to add more and more automated features and advisors to assist with diagnosing and fixing problems. Given all this automation and advice Oracle is providing, a question to ask is whether or not all this advice is "dumbing us down" or "smartening us up". Too often more automation can lead to using these features as a crutch and bypassing the acquisition of key skills and knowledge. Ideally, these features should be tools to help us do our jobs better instead of simply letting Oracle do it for us. The key is to know what help the advisors can provide and know how to use them properly and effectively. The risk is becoming trained to only follow the advisors output, but when a situation arises where our skills and knowledge are needed, it's not there. This presentation will review several cases where the Oracle SQL Tuning Advisor (in particular) isn't able to provide any help with identifying and correcting poorly performing SQL. We'll then review the knowledge and critical thinking skills needed by the DBA or Developer to understand how to make the needed improvements to the code to provide enhanced performance.

1. Are you a monkey or an
astronaut?
The Oracle Advisors from
a different perspective.
Karen Morton
karen.morton@method-r.com

2. Your speaker…
• Karen Morton
– Educator, DBA, developer,
consultant, researcher, author,
speaker, …
– Director
– Software, education, consulting Committed to genuinely satisfying
software performance
• Come see us…
– method-r.com
– karenmorton.blogspot.com
– An Oracle user group near you

3. Agenda
• Introduction
• The “Self-Managing” Database
• Before and After
• What do we really want?
• When the Advisors mislead
• It takes human intelligence
• Conclusion
• Q&A

4. It is impossible to underrate
human intelligence – beginning
with one’s own.
Henry Brooks Adams

5. Introduction

6. Do you remember?

7. What about these guys?

8. Monkeys were first

9. Are you a monkey?

10. Or, are you an astronaut?

11. The Self-Managing
Database

12. The Oracle Advisors

13. Database performance optimization is
a “complex, time consuming task and
requires expensive specialist
consulting skills” to perform

14. Performance diagnosis and tuning
is mostly about following a logical,
methodical approach. Fortunately
computers are pretty good at that!
Graham Wood & Kyle Hailey

15. What role will we choose
and
what is the risk?

17. DBA
x
The system is slow!
!
Please fix it now
x

18. How would you diagnose the problem?

19. 1) Check server to determine if adequate
resources are available

20. 2) Check database and notice many sessions
are waiting on “latch free” waits

21. 3) Drilling down into latches shows most are on
“library cache” and “shared pool” latches

22. 4) From knowledge, experience and reference
books, you know these latches are associated
with hard parsing issues

23. 5) Double-check the hypothesis that hard
parsing is the problem by looking at
the rate at which “parse time elapsed”
and “parse time cpu” are increasing

24. 6) Now determine if one or more sessions are
responsible for the majority of the hard
parses or check shared pool for existence
of many similar statements with the
same SQL plan

25. 7) By looking at the shared pool, you find there
are a small number of plans each of which
has many different SQL texts associated
with it (they differ only by a literal)

26. 8) Now the problem is identified and
you can move on to fixing it

27. With ADDM, how would the problem
have been diagnosed?

28. ADDM says…
SQL statements were not shared due
to usage of literals. This
resulted in additional hard parses
which were consuming significant
database time.

29. ADDM recommends…
ACTION: Investigate application
logic for possible use of bind
variables instead of literals.
Alternatively, you may set the
parameter CURSOR_SHARING to FORCE.

30. …because…
RATIONALE: SQL statements with
PLAN_HASH_VALUE 3106087033 were
found to be using literals. Look
in V$SQL for examples of such SQL
statements.

31. Without ADDM With ADDM
• Multiple investigative steps must be • Consult the latest ADDM report
completed • No mention of latching at all in the
• Must have knowledge of what latch report
free waits indicate • Exact SQL identified with
• Must locate the problem SQL PLAN_HASH_VALUE
• Requires executing queries against • No additional overhead needed as
database that add overhead to the report is based on previously
system collected snapshot data
Given this example, which method
of diagnosis would you prefer?

32. ADDM did a great job!

33. Who wouldn’t want to spend
less time getting to the root
cause of a problem?

34. But…

35. Could you have diagnosed
the problem without ADDM?

36. If you could have diagnosed
the problem, would using ADDM
have been quicker
(and therefore, better)?

37. What would you have done if you
fixed the problem as described
by ADDM, but the user says
“it’s still slow”?

38. Instead of using the human brain,
humans have created another brain
to do the job. There is nothing
around us that is not built to cater
to the needs of the human race and
to ease off the duties of what the
human brain is supposed to do; for
instance, thinking. Is laziness going
to be the death of the human race?
Kurt Vonnegut

39. What do we really want?

40. A magic formula?

41. A checklist, a “how-to”, a tool
or some secret parameter?
alter system set fast = true;

42. What most people really want is
knowledge and experience
…that allows us to confidently
approach a problem and
solve it efficiently

43. We want training that teaches us
how and why things work as they do
…and teaches us how to “know”
what to do and how to do it
when problems arise

44. The part of intuition that involves pattern
matching and recognition of familiar and
typical cases can be trained. If you want people
to size up situations quickly and accurately,
you need to expand their experience base. One
way is to arrange for a person to receive more
difficult cases.
Gary Klein
…or simply have them become an Oracle DBA
or developer.

45. When the advisors
mislead

46. What would you do if the problem
isn’t resolved via the advisors?

47. The advisors aren’t true artificial intelligence

48. They can only pattern match,
aggregate and assimilate answers
based on programmed heuristics

49. They can only suggest solutions
within a realm of known
(i.e. built-in) possibilities

50. They can’t rewrite code

51. They can miss something or “get it wrong”

52. It takes human intelligence

53. SQL Tuning Advisor vs. Human

54. Example

57. So, how did the SQL Advisor do?

58. Let’s compare using AUTOTRACE
Elapsed: 00:00:01.51
--------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
--------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 160 | 2560 | 191 (78)| 00:00:03 |
| 1 | SORT UNIQUE | | 160 | 2560 | 191 (78)| 00:00:03 |
| 2 | UNION-ALL | | | | | |
|* 3 | FILTER | | | | | |
| 4 | HASH GROUP BY | | 10 | 160 | 48 (11)| 00:00:01 |
|* 5 | HASH JOIN OUTER | | 12890 | 201K| 44 (3)| 00:00:01 |
| 6 | INDEX FAST FULL SCAN| CUST2_PK | 1000 | 5000 | 2 (0)| 00:00:01 |
| 7 | TABLE ACCESS FULL | ORD2 | 12890 | 138K| 41 (0)| 00:00:01 |
|* 8 | FILTER | | | | | |
| 9 | HASH GROUP BY | | 50 | 800 | 48 (15)| 00:00:01 |
| 10 | NESTED LOOPS | | 12890 | 201K| 44 (7)| 00:00:01 |
| 11 | TABLE ACCESS FULL | ORD2 | 12890 | 138K| 41 (0)| 00:00:01 |
|* 12 | INDEX UNIQUE SCAN | CUST2_PK | 1 | 5 | 0 (0)| 00:00:01 |
|* 13 | FILTER | | | | | |
| 14 | HASH GROUP BY | | 50 | 800 | 48 (15)| 00:00:01 |
| 15 | NESTED LOOPS | | 12890 | 201K| 44 (7)| 00:00:01 |
| 16 | TABLE ACCESS FULL | ORD2 | 12890 | 138K| 41 (0)| 00:00:01 |
|* 17 | INDEX UNIQUE SCAN | CUST2_PK | 1 | 5 | 0 (0)| 00:00:01 |
|* 18 | FILTER | | | | | |
| 19 | HASH GROUP BY | | 50 | 800 | 48 (15)| 00:00:01 |
| 20 | NESTED LOOPS | | 12890 | 201K| 44 (7)| 00:00:01 |
| 21 | TABLE ACCESS FULL | ORD2 | 12890 | 138K| 41 (0)| 00:00:01 |
|* 22 | INDEX UNIQUE SCAN | CUST2_PK | 1 | 5 | 0 (0)| 00:00:01 |
--------------------------------------------------------------------------------------

59. Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
39419 consistent gets
0 physical reads
0 redo size
26289 bytes sent via SQL*Net to client
1107 bytes received via SQL*Net from client
68 SQL*Net roundtrips to/from client
1 sorts (memory)
0 sorts (disk)
1000 rows processed

60. Elapsed: 00:00:01.50
-------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 160 | 2560 | 184 (77)| 00:00:03 |
| 1 | UNION-ALL | | | | | |
|* 2 | FILTER | | | | | |
| 3 | HASH GROUP BY | | 10 | 160 | 46 (7)| 00:00:01 |
|* 4 | HASH JOIN OUTER | | 12890 | 201K| 44 (3)| 00:00:01 |
| 5 | INDEX FAST FULL SCAN| CUST2_PK | 1000 | 5000 | 2 (0)| 00:00:01 |
| 6 | TABLE ACCESS FULL | ORD2 | 12890 | 138K| 41 (0)| 00:00:01 |
|* 7 | FILTER | | | | | |
| 8 | HASH GROUP BY | | 50 | 800 | 46 (11)| 00:00:01 |
| 9 | NESTED LOOPS | | 12890 | 201K| 44 (7)| 00:00:01 |
| 10 | TABLE ACCESS FULL | ORD2 | 12890 | 138K| 41 (0)| 00:00:01 |
|* 11 | INDEX UNIQUE SCAN | CUST2_PK | 1 | 5 | 0 (0)| 00:00:01 |
|* 12 | FILTER | | | | | |
| 13 | HASH GROUP BY | | 50 | 800 | 46 (11)| 00:00:01 |
| 14 | NESTED LOOPS | | 12890 | 201K| 44 (7)| 00:00:01 |
| 15 | TABLE ACCESS FULL | ORD2 | 12890 | 138K| 41 (0)| 00:00:01 |
|* 16 | INDEX UNIQUE SCAN | CUST2_PK | 1 | 5 | 0 (0)| 00:00:01 |
|* 17 | FILTER | | | | | |
| 18 | HASH GROUP BY | | 50 | 800 | 46 (11)| 00:00:01 |
| 19 | NESTED LOOPS | | 12890 | 201K| 44 (7)| 00:00:01 |
| 20 | TABLE ACCESS FULL | ORD2 | 12890 | 138K| 41 (0)| 00:00:01 |
|* 21 | INDEX UNIQUE SCAN | CUST2_PK | 1 | 5 | 0 (0)| 00:00:01 |
-------------------------------------------------------------------------------------

61. Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
39419 consistent gets
0 physical reads
0 redo size
24813 bytes sent via SQL*Net to client
1107 bytes received via SQL*Net from client
68 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
1000 rows processed

62. Differences?
• Elapsed time went from 1.51s to 1.50s
• Memory sorts were reduced from 1 to 0
• Query cost reduced from 191 to 184
• Consistent gets stayed the same
I still need better response time
and less resource consumption.
Now what?

63. Let’s give the human a try

64. I rewrite the query…
SQL> get cust_disc_rewrite.sql
1 select c.cust_no, sum(o.total_order_price) tot_orders,
2 (case when sum(o.total_order_price) = 0 then 0
3 when sum(o.total_order_price) <= 100000 then .10
4 when sum(o.total_order_price) <= 500000 then .15
5 when sum(o.total_order_price) > 500000 then .20
6 else 0 end) as disc_rate
7 from customer2 c, ord2 o
8 where c.cust_no = o.cust_no(+)
9* group by c.cust_no

65. …and make a huge difference!
Elapsed: 00:00:00.29
-----------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-----------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1000 | 16000 | 46 (7)| 00:00:01 |
| 1 | HASH GROUP BY | | 1000 | 16000 | 46 (7)| 00:00:01 |
|* 2 | HASH JOIN OUTER | | 12890 | 201K| 44 (3)| 00:00:01 |
| 3 | INDEX FAST FULL SCAN| CUST2_PK | 1000 | 5000 | 2 (0)| 00:00:01 |
| 4 | TABLE ACCESS FULL | ORD2 | 12890 | 138K| 41 (0)| 00:00:01 |
-----------------------------------------------------------------------------------

66. The query uses 191 vs. 39,419 LIOs
and runs in .29 seconds!
Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
191 consistent gets
0 physical reads
0 redo size
26279 bytes sent via SQL*Net to client
1107 bytes received via SQL*Net from client
68 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
1000 rows processed

67. The moral of the story

68. Something else to keep in mind…

69. Tools on the market today do not use a
business-task focused approach
to solve a performance problem

70. The Advisors and Enterprise Manager
mainly monitor system-wide metrics

71. What do you do if your
user’s performance problem
isn’t representative of
the system-wide issues?

72. That’s exactly what
Method R
is all about

73. 1. Determine the business task that is
performing sub-optimally that you want to fix
next.

74. 2. Collect detailed statistics about how this
task spends its time while it is performing
poorly.

75. 3. Execute repairs to reduce the task’s
response time and/or resources consumed,
or prove that it is already as efficient as it
can affordably be made.

76. 4. If problems still exist, then go to step 1.

77. There is a belief that there is a downside to a
business task-focused approach.

78. Many believe it’s too difficult to get access to
the information or people needed.

79. It seems easier to focus on what the “system”
has to tell us in order to find and fix problems.

80. Remember the monkeys?

81. Enter the astronaut.

82. The costs of following the wrong advise
can be substantial.

83. If you are working on the wrong thing,
the real problem remains and
the business continues to suffer.

84. Conclusion

85. Don’t let the advisors –
or any ‘automatic’ feature
become the dictator of your career.

86. Tools are there to facilitate our knowledge,
not replace it.

87. Use the tools wisely but not
as a substitute for your own
lack of knowledge.

88. The bottom line

89. The key is about a thirst for knowledge…
not just the knowledge itself.

90. It’s about how we strive to learn,
understand, and know how and why
things work that makes the difference.

91. It is the fulfillment of the desire to know
that leads us to the knowledge we need
to evolve from monkey to astronaut
and to be able to solve the problems
that seem unsolvable.

92. It takes effort but the result is
that you become a respected,
invaluable resource with the skills
and confidence to impact your
business in a positive way.

95. Thank You!