2. About Me
Sr. Director at Amkor Technology
Involved with ColdFusion for 18
years
Author of the O’Reilly’s books:
Programming ColdFusion
Programming ColdFusion MX
Adobe Community Professional
for ColdFusion
Food Geek
Bed Jumper
Photographer
4. Agenda
What We’ll Cover And What We Wont…
What is caching?
What’s Ehcache?
Caching tags and functions
Any questions from Dan Wilson
Disclaimer
Caching Evolution in ColdFusion
Scale Up vs. Scale Out
Caching Architecture Review
Replicated Cache
BigMemory Go
Distributed Cache
Terracotta Management Console
5. Legal Mumbo Jumbo: The Disclaimer
Much of what you’re about to see makes use of third-party libraries and
applications
Some of it requires you to upgrade versions of libraries that ship with
ColdFusion
There are bugs
There are workarounds
There is no official support from Adobe for much of what you’re about to
see
7. Ehcache Implementation : Caching Evolution
Fast restartability
Terracotta Management
Console
Ehcache 2.6.6
11.0
Bug fixes
cacheRemove now takes an
array for ID
Ehcache 2.8.2
Current Version
8. Scale Up vs. Scale Out
ScaleUp
Scale Out
What about cost?
• Hardware
• Data center space
• Power
• Redundancy
• Software Licensing*
• Operations & Sysadmin
How does this change in the cloud?
More CPU
More RAM
More Machines
9. Caching Architectures
In-Process (L1 Cache)
Operates in the same process
(JVM) as the application server
Limited scalability for 32-bit
systems
Must consider possible number of
cache variants
Max JVM heap size
Can fail-over to disk storage
Fast
Data/object serialization is not
required for memory based in-process
cache
JVM
ColdFusion
Cache
In-Process Cache
10. Caching Architectures
Out-of-Process (L2 Cache)
Operates in its own process,
outside of the application
server’s JVM
Highly scalable on both 32-bit
and 64-bit platforms
Scale out
Utilize spare memory throughout
the network
Slower than in-process caching
Data/objects must be
serialized/deserialized
JVM
ColdFusion
Cache
JVM
Cache
Out-of-Process Cache
10
11. Technology
Ehcache
Java caching API
In-process
Open Source /
Commercial
Built-in to
ColdFusion
BigMemory Go
Off-heap memory
storage for the JVM
Stand Alone
Terracotta
Management
Console
Commercial
Terracotta
Distributed in-
memory caching
array
Out-of-process
Open Source no
longer supported
as of version 3.7.7
(Pairs with
Ehcache 2.6.8)
Commercial: Part
of BigMemory
Max
BigMemory Max
Off-heap memory
storage for the JVM
Distributed
Terracotta
Management
Console
Commercial
12. Caching Architecture
Ehcache can be configured to run:
Local: In-process (Ehcache [+
BigMemory Go])
Replicated: In-process (Ehcache [+
BigMemory Go])
Distributed: In-process and Out-of-
process (Ehcache + BigMemory Max)
ColdFusion
Server
Ehcache
ColdFusion
Server
Ehcache
JVM
JVM
ColdFusion
Server
Ehcache
ColdFusion
Server
Ehcache
JVM
JVM
RMI, JMS,
JGroups,
Terracotta
In-Process Cache
Replication
JVM
ColdFusion
Cache
In-Process Cache
JVM
ColdFusion
Cache
JVM
Cache
Out-of-Process Cache
13. To Help Avoid Confusion…
Terracotta recently retired the Terracotta
Server and discontinued the free versions of
BigMemory Go and BigMemory Max
To avoid confusion, keep this in mind:
Ehcache = In-Process Java Caching API – Comes
built-in with ColdFusion
BigMemory Go = Enterprise Ehcache + Off-Heap
Memory Storage for the JVM on a single server -
Commercial (90 day trial)
BigMemory Max = Enterprise Ehcache + Off-Heap
Memory Storage for the JVM + Distributed In-
Memory Caching via the Terracotta Server Array –
Commercial (90 day trial)
14. Still Confused?
BigMemory Go BigMemory Max
Distributed in-memory data store No (standalone JVM) Yes
In-memory capacity Unlimited (limit based on license) Unlimited (limit based on license)
Number of BigMemory mirrored stripes supported N/A Unlimited (limit based on license)
Number of application clients supported N/A Unlimited (limit based on license)
Fast search Yes Yes
Configurable data consistency guarantees N/A Yes
Terracotta Management Console™ Yes Yes
Fast Restartable Store™ Yes Yes
BigMemory-Hadoop Connector support Yes
In-process off-heap storage Yes Yes
Advanced security (authentication/authorization) N/A Yes
Enterprise support Yes Yes
16. Scalability: Cache Replication (Clustering)
All supported cache types can be
replicated
Each CF server in the cluster
maintains its own in-process cache
Data is replicated among L1 caches
Replication via RMI, JMS, JGroups
or Terracotta
Synchronous or asynchronous
Still hit JVM GC issues/limits
Simple configuration via
ehcache.xml file
ColdFusion
Server
Ehcache
ColdFusion
Server
Ehcache
JVM
JVM
ColdFusion
Server
Ehcache
ColdFusion
Server
Ehcache
JVM
JVM
RMI, JMS,
JGroups,
Terracotta
In-Process Cache
Replication
20
17. Potential Replication Gotchas
Synchronous vs. Asynchronous delivery
Asynchronous replication is the fastest method
Because it’s asynchronous the caller returns immediately
Messages are placed in a queue and batched via RMI as they are processed
Potential for data inconsistency exists
Synchronous
Removes potential for data inconsistency
Slower operation as caller waits for replication to complete before returning
Time To Idle
Inconsistent with replicated caching
Data on some nodes will live longer than on others due to cache usage patterns
Do not use unless you don’t care about inconsistent data across cache nodes
18. Replicated Cache Config Per Node – Step 1
Stop your ColdFusion Server
/lib/{servername}/ehcache.xml
Uncomment:
<cacheManagerPeerProviderFactory
class="net.sf.ehcache.distribution.RMICacheManagerPeerProviderFactory"
properties="peerDiscovery=automatic,
multicastGroupAddress=230.0.0.1,
multicastGroupPort=4446, timeToLive=1"
propertySeparator=","
/>
22. Problems with Application Scalability on the JVM
Slow Applications lead to caching
Large caches lead to latency from Garbage
Collection pauses
GC pauses necessitate JVM tuning
Application usage and data grows
More caching is needed
More JVM tuning is needed
As cache size increases, more heap memory is used
More heap memory usage results in unpredictable
and longer GC pauses
Eventually you hit a wall
Add more JVMs
Increased deployment and management
complexity
Tune! Tune! Tune!
23. The Problem with Memory and the JVM
RAM is outpacing the JVM
32GB is now fairly standard on most servers
Amazon EC2 allows for up to 68.4GB
Most of that memory is used inefficiently if it’s used at all
Cached data ages differently than standard business objects
which can confuse the Java garbage collector
The cache expiration determines when the data becomes garbage
And then there's tuning…
25. BigMemory to the Rescue
Cache huge amounts of data with no GC pauses
Easy to implement
Pure Java implementation
Works with all JVMs
Works with both standalone and distributed caches
No application code changes necessary to implement
Avoid GC because of pauses
Off heap store on direct memory buffers
Use all available memory on your server (if you want to)
1 million puts per second
1 line of config to use it
JVM doesn’t have to search for garbage because we already know when a cache item
needs to be thrown out (cache expiry)
27. Upgrade Ehcache and Install BigMemory Go
BigMemory Go requires a newer
version of Ehcache
Adobe doesn’t officially support
upgrading. Do so at your own risk!
There’s currently a bug you need to
be aware of in CF 10 (as of updater
11):
3339491: After upgrading Ehcache,
ehcache.xml is ignored. You’ll need to use
auth-ehcache.xml for now. (Marked as Fixed,
but not yet deployed in a hotfix)
60
28. Upgrade Ehcache and Install Big Memory: Step 1
Stop your ColdFusion server
Rename the following files in your servername/lib directory:
ehcache-core-2.5.1.jar ehcache-core-2.5.1.jar.original
slf4j-api-1.5.6.jar slf4j-api-1.5.6.jar.original
slf4j-log4j12-1.5.6.jar slf4j-log4j12-1.5.6.jar.old
29. Upgrade Ehcache and Install Big Memory: Step 2
Download BigMemory Trial (It will give you Max even if you select Go):
http://terracotta.org/products/bigmemorygo
Extract the following files into your servername/lib directory:
From /apis/ehcache/lib
ehcache-ee-<version>.jar
slf4j-api-<version>.jar
From /common/lib
bigmemory-<version>.jar
Copy the terracotta-license.key file you received by email after downloading
BigMemory into your servername/lib directory
30. Upgrade Ehcache and Install Big Memory: Step 3
Download the Slf4j version bundle that matches the version in your
BigMemory download from http://www.slf4j.org/dist/
Extract the slf4j-log4j12-<version>.jar into your servername/lib directory
31. Upgrade Ehcache and Install Big Memory: Step 4
ehcache.xml (auth-cache.xml):
<cache name="sample-offheap-cache"
maxElementsInMemory="100000"
eternal="true"
memoryStoreEvictionPolicy="LRU"
overflowToOffHeap="true"
maxMemoryOffHeap="2G"/>
Start your ColdFusion server
JVM Config in ColdFusion Admin:
-XX:MaxDirectMemorySize=3G
Restart your ColdFusion server
36. Scalability: Distributed Caching (Commercial)
Commercial license
Runs out-of-process
Tiered caching
Hottest data in L1 (< 1 μs access)
Everything in L2 (< 2 ms access)
"Snap-in scale"
Works with single node Ehcache or
replicated Ehcache
Simple config via ehcache.xml
High data capacity: 2TB+
Highly available
Enterprise
Ehcache
Terracotta
Server
Array
Terracotta
Server Array
ColdFusion
Enterprise
Ehcache
ColdFusion
…
Terracotta
Server
Array
Terracotta
Server Array
37. • Stop your ColdFusion server
• Rename the following files in your
servername/lib directory:
ehcache-core-2.5.1.jar ehcache-core-
2.5.1.jar.original
slf4j-api-1.5.6.jar slf4j-api-1.5.6.jar.old
slf4j-log4j12-1.5.6.jar slf4j-log4j12-1.5.6.jar.old
• Add the following jar files:
From apis/ehcache/lib
ehcache-ee-<version>.jar
slf4j-api-<version>.jar
From apis/toolkit/lib
terracotta-toolkit-runtime-ee-<version>.jar
Ehcache
Terracotta
Server
ColdFusion
Ehcache
ColdFusion
…
Distributed Cache Configuration: Step 1/3
39. Start ColdFusion
JVM Config in ColdFusion Admin:
-X:MaxDirectMemorySize=3G
Stop ColdFusion
Start the Terracotta server, then restart
ColdFusion
bin/start-tc-server.sh
bin/start-tc-server.bat
Ehcache
Terracotta
Server
ColdFusion
Ehcache
ColdFusion
…
Distributed Cache Configuration: Step 3/3
40. Terracotta Gotchas
The Terracotta server must be running before you bring up your ColdFusion
server, otherwise your page request (that tries to do a cache operation) will
spin until the connection times out and an error is returned
Terracotta can only be used with ColdFusion Enterprise. There’s a bug if you
try to use it with the Developer Edition where ColdFusion will return an error
telling you it’s an Enterprise Only feature.
42. Installing the Terracotta Management Console
1. Ensure JAVA_HOME is set
2. Extract the BigMemory go files
3. Start the monitor server: /management-console/lib/start-tmc.bat
4. Configure ehcache (auth_cache.xml) and add monitoring line:
<managementRESTService enabled="true" bind="0.0.0.0:9888" />
5. Open the web browser to http://localhost:9889/tmc