The document describes the evolution of coverage collection from a file-based system to a database-based system and finally to a memory-resident database to address bottlenecks. The memory-resident database stores the coverage database entirely in RAM, uses multithreaded processing and locking to allow high parallelism. This provides major performance improvements like reducing client access time from 30 minutes to 2 seconds and disk I/O by 88x, enabling unlimited parallel simulations.

2. <Insert Picture Here>
RDBMS-based Coverage Collection and Analysis
James Roberts

3. Coverage-driven Flow
sim
sim coverage
sim coverage
coverage
merge
merge
merge
Vault
Grading
Analysis

4. Coverage-driven Flow
sim
sim coverage
sim coverage
coverage
merge
merge
merge
Oracle
Grading
Analysis

5. Baseline: File-based flow
sim
sim coverage
log
sim coverage
log
coverage
log
merge
merge
merge
...
- Each sim writes out a log of its hit coverage
- Each log is diff-and-merged with the vault
Vault file
FLock
bottleneck
- BAD:
- The vault can easily exceed 200,000 coverage objects and 300 Megabytes
- Diff-and-merge can exceed 10 minutes just for a single sim
- Only one sim can hold the file lock on the vault at one time
- These can be thousands of sims in parallel
- The vault is rewritten in its entirety after every merge
- that means non-stop disk activity!

6. Coverage using a database
sim
sim SQL
sim SQL
SQL
...
- Instead of a flat file, the vault is an Oracle database
- Instead of coverage logs, they are SQL queries to the database
Oracle
- GOOD:
- Each sim no longer requires a file lock on the entire vault
- Merges can be parallel
- Oracle database server is already very familiar with this kind of
multiple-parallel-transaction situation
- Only subset of coverage actually touched by sim is diff-and-merged
- A rich set of SQL commands is available to do diag & coverage analysis
on the vault
- 3X speedup over file-based flow (and it's parallel)

7. A relational database table
Vault
CId Name Block
l2_hit_after_miss
ack_stg_2
clk_off
bit1_carry_bit0
...
(lock)Hits
0->1
0
1000
1
200,000
Row-levellocking
SERDES
L2
PADIO
ALU
2
1
3
4

8. Bottleneck: Oracle database flow
sim
sim
sim
...
+ Oracle database server is multithreaded, and recognizes when coverage
objects are unrelated and can be merged in parallel
- The fundamental problem remains, though: thousands of simulations
contending for access to a single disk
+ We address this later by moving the merge to RAM
Oracle
SQL
SQL
SQL

9. Oracle database flow
220,000 objects accessed per second,
yet the entire database only has 170,000 objects

10. Database activity
- Database server is bumping against a wall at around
1600 statements/second

11. The Line Grows Longer
- As the database server cannot keep up with requests,
the line of simulations grows longer and longer
- Workaround: throttle your simulations to acceptable levels
The line has grown 100 deep!
+ We’re achieving a parallelism of 20. (this is good)

12. Memory-resident database
- Store main coverage database 100% in the RAM
- Multithreaded RAM database server
- Each simulation is a client
...
Disk
Thread 1
Thread 2
Thread N'
...
Client 1
RAMdatabase
Thread 0
dirty
dirty
locks
Client 2
Client 3
Client N

13. Memory-resident database
- simulations connect to server via TCP sockets, RPC
- coverage data transmitted directly to server
- no disk involved
...
Disk
Thread 1
Thread 2
Thread N'
...
Client 1
RAMdatabase
Thread 0
dirty
dirty
locks
Client 2
Client 3
Client N

14. Memory-resident database
- Locking in RAM database to prevent contention across threads
- Dirty bits to flag updates
- Updating dirty (but unlocked) objects perfectly legal
...
Disk
Thread 1
Thread 2
Thread N'
...
Client 1
RAMdatabase
Thread 0
dirty
dirty
locks
Client 2
Client 3
Client N

15. Memory-resident database
- Dedicated thread to flush updates to disk
- Clears dirty bits
+ Disk never gets overloaded
...
Disk
Thread 1
Thread 2
Thread N'
...
Client 1
RAMdatabase
Thread 0
dirty
dirty
locks
Client 2
Client 3
Client N

16. Memory-resident database
- Disk flush is completely decoupled from merge
...
Disk
Thread 1
Thread 2
Thread N'
...
Client 1
RAMdatabase
Thread 0
dirty
dirty
locks
Client 2
Client 3
Client N
+ Client access time reduced from 30 minutes
to 2 seconds!!! (900X speedup)

17. Disk activity w/ Memory-resident DB
- Disk traffic reduced from 1600 statements/second to
18 statements/second
- 88X reduction in disk load!!!

18. Before: Non-Memory-Resident DB
- Compare with old results:
1600 statements/second vs. 18 statements/second

19. Disk I/O – 90% coverage population
Day 1 Day 2 Day 3 Day 4 Day 5
0
500
1000
1500
2000
2500
3000
3500
4000
4500
- RAM cache filters out all the read accesses
- at >90% coverage population, most database traffic is read-only
- 15030 disk writes over 5 days: or 1 disk I/O every 165 seconds
+ Disk use is infinitesimal at >90% coverage

20. Conclusions
Merge no longer the bottleneck in coverage flow
Unlimited simulations, where previously we had
to throttle it
88X disk I/O reduction worst-case
Virtually zero I/O at >90% coverage
30-minute latency reduced to 2 seconds
> 4-week uptimes for RAM server process
Results have been positive to the point that
people sometimes question our data

21. <Insert Picture Here>
Metrics