The document discusses performance testing of Firebird databases. It provides guidelines for testing including understanding the goal of the test, hardware and software setup, and accounting for caching effects. Testing covered analyzing queries on different Firebird versions and settings, comparing the use of Firebird cache versus OS cache, loading speed with different numbers of concurrent users and database sizes, and testing concurrency levels between Classic and SuperClassic architectures. Results showed improvements in Firebird 2.5 and with SuperClassic architecture over Classic.

1. Firebird
Performance Testing
19-OCT-2010
Dmitry Yemanov
dimitr@firebirdsql.org
Alexey Kovyazin, IBSurgeon
test@ib-aid.com

2. Basic considerations
● Learn the used test in details. Make sure you
understand its goal. Figure out what is actually
being tested.
● Pay attention to the hardware and software
setup. Avoid exotic configurations.
● Take caching effects and differencies among
runs into account. Remove extreme values from
the results or comment them appropriately.
● Don't compare apples to oranges.

3. Firebird settings
● Architecture: Classic or SuperServer.
● Database page size.
● Page cache size. Remember about the
filesystem cache.
● Sort buffer size. It differs between CS and SS.
● Lock table settings: LockMemSize,
LockHashSlots.
● Forced writes setting.
● Garbage collection policy.

4. Testing analytic queries
● Single user
● Involves optimizer decisions
and raw execution efficiency
● Hardware: Intel Core2 2.4GHz Quad, 4GB
RAM, SATA2 HDD 7200rpm
● Firebird settings: Classic Server, 256 buffer
pages
● Results are in seconds, the less the better

5. Total time spent
2500
FB 2.1 : 2052 s
FB 2.5 : 271 s 2000
1500
FB 2.1
FB 2.5
1000
500
0
1

6. Time distribution among tests
2000
1800
1600
1400
1200
1000 FB 2.1
FB 2.5
800
600
400
200
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

7. FB cache vs OS cache
● Database is 1.5GB
● 1.5GB of Firebird page cache and disabled OS
filesystem cache
● 1MB of Firebird page cache and enabled OS
filesystem cache
● Results are in seconds, the less the better

8. Total time spent
800
FB cache : 709 s
OS cache : 271 s
700
600
500
400 FB cache
OS cache
300
200
100
0
1

9. Time distribution among tests
400
350
300
250
200 FB cache
OS cache
150
100
50
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

10. Load speed test
● 4 concurrent users
● Database sizes: 9Gb and 22Gb
● Hardware: Intel Core2 2.4GHz Quad, 4GB
RAM, SATA2 HDD 7200rpm
● Firebird settings: Classic vs SuperClassic,
256 buffer pages
● Firebird versions: 2.1 Classic, 2.5 SuperClassic
● Results are seconds, the less the better

11. Classic vs SuperClassic Load
2000
Database 9Gb loading: 1800
1600
1400
2.1 CS : 1759 sec 1200
2.5 CS : 1130 sec 1000
800
2.5 SC: 1302 sec 600
400
200
0
2.1 Classic 2.5 Classic 2.5 SuperClassic
5000
Database 22Gb loading: 4500
4000
3500
2.1 CS : 4290 sec 3000
2500
2.5 CS : 3971 sec 2000
2.5 SC: 3328 sec 1500
1000
500
0
2.1 Classic 2.5 Classic 2.5 SuperCalssic

12. Testing concurrency
● Many concurrent users
● Involves raw execution efficiency
as well as overall scalability
● Hardware: Intel Core2 2.4GHz, 4GB RAM,
SATA2 HDD 7200rpm
● Firebird settings: Classic vs SuperClassic,
256 buffer pages
● Results are in completed transactions per
minute, the more the better

13. Classic vs SuperClassic
350
100 users:
300
CS : 282 tpm 250
SC : 313 tpm 200
CS
250 users:
SC
150
100
CS : 159 tpm
SC : 181 tpm
50
0
100 users 250 users

14. Thank you!
Questions?
test@ib-aid.com