1. Copyright
©
2014
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Understanding
Oracle
GoldenGate
Architecture
Bobby
CurCs
Senior
Technical
Consultant
Accenture
Enkitec
Group
July
9,
2014
1
Performance
Tuning
to
the
OS
Level

2. Speaker
Info
• Douglasville,
Georgia
• Senior
Technical
Consultant
• Expert
OEM12c
(2013)/PracCcal
ODA
(2014)
• TwiUer:
@dbasolved
• Blog:
hUp://dbasolved.com
• Email:
bcurCs@enkitec.com
curCsbl@gmail.com

3. Accenture
Enkitec
Group
Enkitec
has
joined
Accenture’s
Infrastructure
Services
as
Accenture
Enkitec
Group
• 17,000
Infrastructure
Services
professionals
• 52,000
Oracle
professionals
Focus
on
Oracle
Engineered
Systems
SoluCons
• Database
MigraCons
&
TransformaCons
• Database-­‐as-­‐a-­‐Service
• Oracle
ApplicaCons
on
Engineered
Systems
• Cloud-­‐based
soluCons
leveraging
Engineered
Systems

4. Agenda
• General
InformaCon
• What
is
GoldenGate
• Benefits
• Basic
Architecture
• Use
Cases
• Tuning
InformaCon
• GoldenGate
Tuning
• OperaCng
System
Tuning
• Processes
• Memory
• Disk

5. General
InformaCon

6. What
is
Oracle
GoldenGate
• Comprehensive
sobware
for
real-­‐Cme
data
integraCon
in
heterogeneous
environments
• Benefits
• High
Availability
SoluCons
• Real-­‐Time
Data
IntegraCon
• TransformaCon
of
Data
• TransacConal
Change
Data
Capture
• Data
ReplicaCon
• VerificaCon
of
Data
(Veridata)
•
Primary
ReplicaCon
Tool
in
Oracle
Database
12c
(12.1.0.1)
• Oracle
Upgrade
Guide,
SecCons
8.1.6
&
8.1.7
• MulCple
ways
to
successfully
monitor
Oracle
GoldenGate

7. Benefits
• Real-­‐Time
Data
Capture
• Near-­‐Zero
DownCme
migraCons
• Quickly
upgrade
databases
• Open
Architecture
(Heterogeneous)
• Disaster
Recovery
• Data
DistribuCon
• Distributed
Offices
• Query
Off-­‐loading
• ReporCng
Instances

8. Basic
Architecture
Capture:
commiUed
transacCons
are
captured
(and
can
be
filtered)
as
they
occur
by
reading
the
transacCon
logs.
Trail:
stages
and
queues
data
for
rouCng.
Pump:
distributes
data
for
rouCng
to
target(s).
Route:
data
is
compressed,
encrypted
for
rouCng
to
target(s).
Delivery:
applies
data
with
transacCon
integrity,
transforming
the
data
as
required.
Source
Oracle / Non-Oracle
Database
Target
Oracle / Non-Oracle
Database
Capture
Delivery
Trail
Files
Pump
Trail
Files
Pump
Delivery
Capture
Bi-directional
LAN / WAN / Internet
Over TCP/IP
Trail
Files
Trail
Files
Modular
Architecture

9. Use
Cases
Unidirec8onal
Query
Offloading
Zero-­‐DownCme
MigraCon
Data
IntegraCon
Cloud
or
On-­‐Premise
Bi-­‐Direc8onal
AcCve-­‐AcCve
for
MulC-­‐Master/HA
Cloud
or
On-­‐Premises
Data
Distribu8on
via
Messaging
Cloud
Apps
Integra8on
Big
Data
Delivery
Real/Time
and
Batch
Delivery
Structured
Data
to
Data
Reservoir

10. Tuning
InformaCon

11. Performance
Tuning
• Per
Oracle
• Baselining
• GoldenGate
Tuning
• Lag
Times
• AWR/ASH
Reports
• CPU
• Memory
• I/O
• OperaCng
System
Tuning

12. Per
Oracle
• MulCple
Process
Groups
• Data
Integrity
• Number
of
Groups
• Dependent
on
amount
of
memory
• Use
Network
Efficiently
• Benchmark
Extract/Replicat
(checkpoints)
•
Eliminate
disk
I/O
• Writes
data
sequenCally,
RAID
0+1
•
Manage
VM
and
Paging
• Cachemgr

13. Baselining
• Should
be
first
thing
you
gather
• What
should
be
gathered?
• GoldenGate
• Lag
• Checkpoint
Info
• General
AWR
InformaCon
(DB)
• OperaCng
System
• CPU
performance
(mpstat)
• Memory
performance
(vmstat)
• I/O
performance
(iostat)

14. GoldenGate
Lag
Info
• ReporCng
Lag
• Gives
an
idea
of
how
long
it
takes
data
to
transfer
on
network
• GGSCI
commands
• lag
extract
<group
name>
• lag
replicat
<group
name>

15. GoldenGate
Checkpoint
Info
• Checkpoint
• Helps
detect
network
boUlenecks
• GGSCI
commands
• info
extract
<group
name>,
showch
<number>

16. Database
Tuning
• AWR
Reports
• Wait
Event:
Streams
Miscellaneous
Event

17. CPU
InformaCon
• Use
mpstat
• mpstat
–P
ALL
[count
[interval]]
• sar
can
be
used
as
well

18. Memory
Performance
• Use
vmstat
• Reports
on
virtual
memory
staCsCcs
• vmstat
–S
M
[count
[interval]]

19. Disk
I/O
Tuning
• Use
iostat
• Reports
staCsCcs
transfered
between
devices
• iostat
–m
[count
[interval]]

20. Baseline
Completed
Metric
Baseline
Running
Lag
5
sec
(ext)/12
sec
(rep)
1
sec
(ext)/4
sec
(rep)
CPU
4.64%
(iowait
(all))
7.73%
(iowait(all))
Memory
2214M
(free)
2109M
(free)
Disk
IO
29.16
tps
29.06
tps
• ObservaCon:
• Serial
processing
(no
parallel)
• Baseline
was
taken
with
higher
transacCon
count
• Disk
I/O
was
slightly
lower
with
high
CPU
waits
• More
memory
was
used
during
the
“running”
test

21. Tune
GoldenGate

22. Tune
GoldenGate
• MulCple
Process
Groups
(Classic)
• Parallel
Replicat
Groups
• 3
to
1
raCo
(replicat
to
trail
(classic))
• Reduced
with
Integrated
Replicat
• DB
(11.2.0.4)
and
OGG
12c
(12.1.2)
Source
Oracle
&
Non-­‐Oracle
Database(s)
Target
Oracle
&
Non-­‐Oracle
Database(s)
Capture Trail
Files
Pump DeliveryTrail
Files
Delivery
Delivery
LAN / WAN / Internet
Over TCP/IP

23. Tune
GoldenGate
• MulCple
Process
Groups
(Classic)
• Parallel
Extract
Groups
• 1
to
1
raCo
(extract
to
trail
to
replicat)
• Reduced
with
Integrated
processes
Source
Oracle
&
Non-­‐Oracle
Database(s)
Target
Oracle
&
Non-­‐Oracle
Database(s)
Trail
Files
Pump Trail
Files
Delivery
Delivery
Delivery
LAN / WAN / Internet
Over TCP/IP
Capture
Capture
Capture

24. Tune
GoldenGate
• Use
Network
Efficiently
• Lag
Cmes
increase
• Check
target
database
performance
• Consult
Network
Team
for
any
performance
issue
• Checkpoint
Extract/Replicat
• send replicat rep, status
• Indicates
any
delay
with
replicat

25. Tune
GoldenGate
• Eliminate
disk
I/O
• Try
to
eliminate
this
first
• System
ConfiguraCons
• Most
shops,
disk
are
controlled
by
SAN
Admin
or
internal
devices
• Choose
RAID
0+1
over
RAID
5
• GoldenGate
ConfiguraCon
• Increase
values
of
parameters:
• CHECKPOINTSECS
• GROUPTRANSOPS
• EOFDELAY or EOFDELAYSECS

26. Tune
GoldenGate
• Manage
VM
and
Paging
• Default:
use
as
much
memory
allowed
• Default:
./dirtmp
and
as
much
space
as
allowed
• Can
be
controlled
with
cachemgr
parameter
• Valid
for
Extract/Replicat
• Current
seyngs
can
be
viewed
in
report
file

27. Tune
GoldenGate
(cachemgr
con’t)
• EsCmate
swap
space
needed
• Extract_Swap_Needed =(PROCESS_VM *
num_extracts) + (swap_other_processes)
• Replicat_Swap_Needed =(PROCESS_VM *
num_replicats) + (swap_other_processes)
•
Review
in
report
file
• send [extract/replicat] <group name>,
cachemanager

28. Profiling
GoldenGate

29. Profiling
the
GoldenGate
Processes
• perf
• Used
to
profile
OS
processes
and
hardware
performance
• strace
• Used
to
see
what
processes
are
calling
• backtrace
file

30. Profiling
the
GoldenGate
Extract
• backtrace
file
output
• InteresCng
find
with
extract
process
(6
Threads
=
1
extract)

31. Profiling
the
GoldenGate
Extract
• Threads
IdenCfied
for
extract
• Number
of
threads
increase/add
extracts
• 1
extract
=
6
threads
• 2
extracts
=
12
threads,
etc..
• Each
thread
takes/uses
memory
• Average
Memory
22-­‐50
MB
• 3.6-­‐8.3
MB
per
thread

32. Conclusion
• Reviewed
Oracle
GoldenGate
Architecture
• Discussed
tuning
aspects
of
Oracle
GoldenGate
for
applicaCon
and
operaCng
system
• Quickly
looked
at
trace
of
an
extract
process

34. Speaker
Info
• Douglasville,
Georgia
• Senior
Technical
Consultant
• Expert
OEM12c
(2013)/PracCcal
ODA
(2014)
• TwiUer:
@dbasolved
• Blog:
hUp://dbasolved.com
• Email:
bcurCs@enkitec.com
curCsbl@gmail.com

35. Copyright
©
2014
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Understanding
Oracle
GoldenGate
Architecture
Bobby
CurCs
Senior
Technical
Consultant
Accenture
Enkitec
Group
July
9,
2014
Oracle
ConfidenCal
–
Internal/Restricted/
Highly
Restricted
35
Performance
Tuning
to
the
OS
Level