Weblogic Cluster Performance Tuning

1. Performance Tuning in Weblogic Server

2. Performance Tuning Topics
● Tuning EJB
● Tuning JMS
● Tuning Web Application
● Tuning Web Services
● Tuning JDBC
● Tuning Server

3. Understand Your Performance
Objectives
To determine your performance objectives, you need to understand the application
deployed and the environmental constraints placed on the system. Gather information
about the levels of activity that components of the application are expected to meet, such
as:
• The anticipated number of users.
• The number and size of requests.
• The amount of data and its consistency.
• Determining your target CPU utilization.
Your target CPU usage should not be 100%, you should determine a target CPU
utilization based on your application needs, including CPU cycles for peak usage. If
your CPU utilization is optimized at 100% during normal load hours, you have no
capacity to handle a peak load. In applications that are latency sensitive and
maintaining the ability for a fast response time is important, high CPU usage
(approaching 100% utilization) can reduce response times while throughput stays
constant or even increases because of work queuing up in the server. For such
applications, a 70% - 80% CPU utilization recommended. A good target for non-

4. Understand Performance Objective
• The configuration of hardware and software such as CPU type, disk size
vs. disk speed, sufficient memory.
There is no single formula for determining your hardware requirements. The
process of determining what type of hardware and software configuration is
required to meet application needs adequately is called capacity planning.
Capacity planning requires assessment of your system performance goals
and an understanding of your application. Capacity planning for server
hardware should focus on maximum performance requirements.
• The ability to interoperate between domains, use legacy systems,
support legacy data.
• Development, implementation, and maintenance costs.

5. Monitor Utilization
– Monitor Disk and CPU Utilization
Run your application under a high load while monitoring the:
• Application server (disk and CPU utilization)
• Database server (disk and CPU utilization)
The goal is to get to a point where the application server achieves your target CPU
utilization. If you find that the application server CPU is under utilized, confirm whether
the database is bottle necked. If the database CPU is 100 percent utilized, then check
your application SQL calls query plans. For example, are your SQL calls using indexes
or doing linear searches? Also, confirm whether there are too many ORDER BY clauses
used in your application that are affecting the database CPU.
If you discover that the database disk is the bottleneck (for example, if the disk is 100
percent utilized), try moving to faster disks or to a RAID (redundant array of
independent disks) configuration, assuming the application is not doing more writes then
required.

6. Monitor Utilization
– Monitor Disk and CPU Utilization
.
Once you know the database server is not the bottleneck, determine whether the
application server disk is the bottleneck. Some of the disk bottlenecks for application
server disks are:
• Persistent Store writes
• Transaction logging (tlogs)
• HTTP logging
• Server logging
The disk I/O on an application server can be optimized using faster disks or RAID,
disabling synchronous JMS writes, using JTA direct writes for tlogs, or increasing the
HTTP log buffer.

7. Monitor Data Transfers Across the
Network
– Locate Bottlenecks in Your System
If you determine that neither the network nor the database server is the bottleneck, start
looking at your operating system, JVM, and WebLogic Server configurations. Most
importantly, is the machine running WebLogic Server able to get your target CPU
utilization with a high client load? If the answer is no, then check if there is any locking
taking place in the application. You should profile your application using a
commercially available tool (for example, JProbe or OptimizeIt) to pinpoint bottlenecks
and improve application performance.
– Minimize Impact of Bottlenecks
In this step, you tune your environment to minimize the impact of bottlenecks on your
performance objectives. It is important to realize that in this step you are minimizing the
impact of bottlenecks, not eliminating them. Tuning allows you to adjust resources to
achieve your performance objectives.

8. EJB Tuning
■Deploymentdescriptorsareschema-based. Descriptorsthatarenew inthisreleaseof
WebLogicServer arenotavailableasDTD-baseddescriptors. ■Avoidusingthe
RequiresNew transactionparameter. UsingRequiresNew causestheEJB container to
startanew transactionafter suspendingany currenttransactions. Thismeansadditional
resources, includingaseparatedatabaseconnectionareallocated.
■Uselocal-interfacesor setcall-by-referencetotruetoavoidtheoverheadof
serializationwhenoneEJB callsanother or anEJB iscalledby aservlet/JSP inthesame
application. Notethefollowing:
– Inreleaseprior toWebLogicServer 8.1, call-by-referenceisturnedonby default. For
releasesof WebLogicServer 8.1andhigher, call-by-referenceisturnedoff by default.
Older applicationsmigratingtoWebLogicServer 8.1andhigher thatdonotexplicitly
turnoncall-by-referencemay experienceadropinperformance.
– Thisoptimizationdoesnotapply tocallsacrossdifferentapplications.

9. EJB Tuning
■UseStatelesssessionbeansover Stateful sessionbeanswhenever possible. Stateless
sessionbeansscalebetter thanstateful sessionbeansbecausethereisnostate
informationtobemaintained.
■WebLogicServer providesadditional transactionperformancebenefitsfor EJBsthat
resideinaWebLogicServer cluster. WhenasingletransactionusesmultipleEJBs,
WebLogicServer attemptstouseEJB instancesfromasingleWebLogicServer
instance, rather thanusingEJBsfromdifferentservers. Thisapproachminimizes
network trafficfor thetransaction. Insomecases, atransactioncanuseEJBsthatreside
onmultipleWebLogicServer instancesinacluster. Thiscanoccur inheterogeneous
clusters, whereall EJBshavenotbeendeployedtoall WebLogicServer instances. In
thesecases, WebLogicServer usesamultitier connectiontoaccessthedatastore, rather
thanmultipledirectconnections. Thisapproachusesfewer resources, andyieldsbetter
performancefor thetransaction. However, for bestperformance, thecluster shouldbe
homogeneous
— all EJBsshouldresideonall availableWebLogicServer instances.

10. Tuning Stateful Session Bean Cache
TheEJB Container cachesstateful sessionbeansinmemory uptoacount
specifiedby themax-beans-in-cache parameter specifiedin
weblogic-ejb-jar.xml. Thisparameter shouldbesetequal tothenumber
of concurrentusers. Thisensuresminimumpassivationof stateful sessionbeans
todisk andsubsequentactivationfromdisk whichyieldsbetter performance.

11. Tuning Entity Bean Cache
Transaction-Level Caching
Onceanentity beanhasbeenloadedfromthedatabase, itisalwaysretrievedfromthecache
whenever itisrequestedwhenusingthefindByPrimaryKey or invokedfromacached
referenceinthattransaction. Gettinganentity beanusinganon-primary key finder always
retrievesthepersistentstateof thebeanfromthedatabase.
Caching between Transactions
Entity beaninstancesarealsocachedbetweentransactions. However, by default, thepersistent
stateof theentity beansarenotcachedbetweentransactions. Toenablecachingbetween
transactions, setthevalueof thecache-between-transactions parameter totrue.
Ready Bean Caching
For entity beanswithahighcachemissratio, maintainingready beaninstancescanadversely
affectperformance.
If youcansetdisable-ready-instances intheentity-cache element of an
entity-descriptor, thecontainer doesnotmaintaintheready instancesincache. If the
feature is enabled in the deployment descriptor, the cache only keeps the active
instances. Once the involved transaction is committed or rolled back, the bean instance is
moved from active cache to the pool immediately

12. MDB Pool Tuning
Thelifecycleof MDBsisvery similar tostatelesssessionbeans. TheMDB pool
hasthesametuningparametersasstatelesssessionbeansandthesamefactors
apply whentuningthem. Ingeneral, mostuserswill findthatthedefaultvalues
areadequatefor mostapplications.

13. Tuning Stateless Session Bean Pool
TheEJB container maintainsapool of statelesssessionbeanstoavoidcreatingand
destroyinginstances. Thoughgenerally useful, thispoolingisevenmoreimportantfor
performancewhentheejbCreate() andthesetSessionContext() methods
areexpensive. Thepool hasalower aswell asanupper bound. Theupper boundisthe
moreimportantof thetwo.
Theupper boundisspecifiedby themax-beans-in-free-pool parameter. It
shouldbesetequal tothenumber of threadsexpectedtoinvoketheEJB concurrently.
Usingtoosmall of avalueimpactsconcurrency.
Thelower boundisspecifiedby theinitial-beans-in-free-pool
parameter. Increasingthevalueof initial-beans-in-free-pool increasesthe
timeittakestodeploy theapplicationcontainingtheEJB andcontributestostartuptime
for theserver. Theadvantageisthecostof creatingEJB instancesisnotincurredatrun
time. Settingthisvaluetoohighwastesmemory.

14. Tuning Entity Bean Pool
Theentity beanpool servestwopurposes:
A targetobjectsfor invocationof findersviareflection.
A pool of beaninstancesthecontainer canrecruitif itcannotfindaninstance
for aparticular primary key inthecache.
Theentity pool containsanonymousinstances(instancesthatdonothavea
primary key). Thesebeansarenotyetactive(meaningejbActivate() has
notbeeninvokedonthemyet), thoughtheEJB contexthasbeenset. Entity bean
instancesevictedfromtheentity cachearepassivatedandputintothepool. The
tunablesaretheinitial-beans-in-free-pool andmax-beans-in-
free-pool. UnlikestatelesssessionbeansandMDBs, themax-beans-in-
free-pool hasnorelationwiththethreadcount. Youshouldincreasethe
valueof max-beans-in-free-pool if theentity beanconstructor or
setEnityContext() methodsareexpensive.

15. JMS Tuning
● Alwaysconfigurequotas
● Verify thatdefaultpagingsettingsapply toyour needs. Paginglowersperformance
butmay berequiredif JVM memory isinsufficient.
● Avoidlargemessagebacklogs.
● Createandusecustomconnectionfactorieswithall applicationsinsteadof using
defaultconnectionfactories, includingwhenusingMDBs. Defaultconnection
factoriesarenottunable, whilecustomconnectionfactoriesprovidemany options
for performancetuning.
● Writeapplicationssothatthey cacheandre-useJMS clientresources, including
JNDI contextsandlookups, andJMS connections, sessions, consumers, or
producers. Theseresourcesarerelatively expensivetocreate. For informationon
detectingwhencachingisneeded, aswell asonbuilt-inpoolingfeatures
●

16. JMS Tuning
● For asynchronousconsumersandMDBs, tuneMessagesMaximum onthe
connectionfactory. IncreasingMessagesMaximum canimproveperformance,
decreasingMessagesMaximum toitsminimumvaluecanlower performance, but
helpsensurethatmessagesdonotendupwaitingfor aconsumer that'salready
processingamessage.
● Avoidsinglethreadedprocessingwhenpossible. Usemultipleconcurrentproducers
andconsumersandensurethatenoughthreadsareavailabletoservicethem.
● Tuneserver-sideapplicationssothatthey haveenoughinstances. Consider creating
dedicatedthreadpoolsfor theseapplications.
● For client-sideapplicationswithasynchronousconsumers, tuneclient-sidethread
pools
● Inparticular, it'snormally bestfor multipleJMS servers, destinations, andother
servicestosharethesamestoresothatthestorecanaggregateconcurrentrequests
intosinglephysical I/O requests, andtoreducethechancethataJTA transaction
spansmorethanonestore. Multiplestoresshouldonly beconsideredonceit'sbeen
establishedthattheasinglestoreisnotscalingtohandlethecurrentload.

17. Tuning Web Application
Disable Page Checks
Youcanimproveperformanceby disablingservletandJDP pagechecks. Seteachof the
followingparametersto-1:
pageCheckSeconds
servlet-reload-check-secs
servlet Reload Check
Thesearedefaultvaluesfor productionmode.
Use Custom JSP Tags
OracleprovidesthreespecializedJSP tagsthatyoucanuseinyour JSP pages: cache,
repeat, andprocess. Thesetagsarepackagedinataglibrary jar filecalledweblogic-
tags.jar. Thisjar filecontainsclassesfor thetagsandataglibrary descriptor (TLD).
Tousethesetags, youcopy thisjar filetotheWebapplicationthatcontainsyour JSPs
andreferencethetaglibrary inyour JSP

18. Tuning Web Application
Precompile JSPs
YoucanconfigureWebLogicServer toprecompileyour JSPswhenaWebApplicationisdeployedor re-
deployedor whenWebLogicServer startsupby settingtheprecompileparameter totrueinthejsp-
descriptor elementof theweblogic.xml deploymentdescriptor. Toavoidrecompilingyour JSPs
eachtimetheserver restartsandwhenyoutargetadditional servers, precompilethemusingweblogic.jspc
andplacethemintheWEB-INF/classesfolder andarchivethemina.war file. Keepingyour sourcefilesina
separatedirectory fromthearchived.war fileeliminatesthepossibility of errorscausedbyaJSP havinga
dependency ononeof theclassfiles.
Disable Access Logging
Settingtheaccess-logging-disabled elementcaneliminateaccessloggingof theunderlying Web
application, which can improve server throughput by reducing the loggingoverhead. Seecontainer-
descriptor inDevelopingWebApplications, Servlets, and JSPs for Oracle WebLogic Server.
Use HTML Template Compression
Usingthecompress-html-template elementcompressestheHTML intheJSP template blocks
which can improve runtime performance. If the JSP's HTML template block containsthe<pre>
HTML tag, donotenablethisfeature. Seejsp-descriptor inDevelopingWebApplications, Servlets,
andJSPsfor OracleWebLogic Server.

19. Tuning Web Services
DesignWebServiceapplicationsfor course-grainedservicewithmoderate
sizepayloads.
■Choosecorrectservice-style& encodingfor your wepserviceapplication.
■Control serializer overheadsandnamespacesdeclarationstoachievebetter
performance.
■UseMTOM/XOP or FastInfosettooptimizingtheformatof aSOAP
message.
■Carefully designSOAP attachmentsandsecurity implementationsfor
minimumperformanceoverheads.

20. Tuning Web Services
■Consider usinganasynchronousmessagingmodel for applicationswith:
– Slow andunreliabletransport.
– Complex andlong-runningprocess.
■For transactional ServiceOrientedArchitectures(SOA) consider usingtheLastLogging
Resourcetransactionoptimization(LLR) toimproveperformance.
■Usereplicationandcachingof dataandschemadefinitionstoimproveperformanceby
minimizingnetwork overhead.
■Consider any XML compressiontechniqueonly whenXML compression/decompression
overheadsarelessthannetwork overheadsinvolved.
■Applicationsthatareheavy usersof XML functionality (parsers) may encounter performance
issuesor runoutof filedescriptors. Thismay occur becauseXML parser instancesare
bootstrappedby doingalookupinthejaxp.propertiesfile(JAXP API). Oraclerecommendssetting
thepropertiesonthecommandlinetoavoidunnecessary fileoperationsatruntimeandimprove
performanceandresourceusage.

21. Tuning Heavily Loaded Systems to Improve
Web Service Performance
Setting the Buffering Sessions
ThereliablemessagingandbufferingfeaturesuseJMS queuesessionstosend
messagestothereliability/buffer queues. By default, WebLogicServer allocates
10sessionsfor bufferingwhichenables10clientstoenqueuemessages
simultaneously ontothereliability/buffer queue.
For asynchronousrequest-response, therequestandresponseportionof the
communicationexchangecountseparately, astwoclients. Inthiscase, the
defaultpool of sessionscansupportfivesimultaneousasynchronousrequest-
responseclients. Toaccommodatethenumber of concurrentclientsyouexpect
inyour application, setthefollowingparameter totwicethenumber of expected
clientthreads:
-Dweblogic.wsee.buffer.QueueSessionPoolSize=size

22. Tuning Heavily Loaded Systems to Improve
Web Service Performance
Releasing Asynchronous Resources
Whenusingtheasynchronousrequest-responsefeature, WebLogicServer persistently
storesinformationabouttherequestuntil theasynchronousresponseisreturnedtothe
client. Theseresourcesremaininthepersistentstoreuntil they arereleasedby a
backgroundthread, calledthestore cleaner.
Often, theseresourcescanbereleasedsooner. Executingthestorecleaner more
frequently canhelptoreducethesizeof thepersistentstoreandminimizethetime
requiredtocleanit. By default, thestorecleaner runsevery twominutes(120000ms).
Oraclerecommendsthatyousetthestorecleaner interval tooneminute(60000ms)
usingthefollowing
Javasystemproperty:
-Dweblogic.wsee.StateCleanInterval=60000