Cache is a component that stores data to serve future requests faster. It provides performance benefits by storing frequently accessed data in memory for quick retrieval. There are different types of caches like CPU caches, browser caches, and database caches. Caching login information can improve performance by retrieving user profiles from cache instead of querying the database on every request. This reduces load on the database server. However, updating cached data whenever the database changes can be challenging and require techniques like retries, batch processing, and global locks to ensure consistency.

1. 초보자를 위한 분산 캐시 이야기
charsyam@naver.com
http://charsyam.wordpress.com

2. 잉여서버개발자 In NHN
관심분야!!!
Cloud, Big Data
특기!!!
발표 날로 먹기!!!

3. What is Cache?

4. Cache is a component
that transparently stores data so
that future requests for that
data can be served faster.
In Wikipedia

5. Cache 는 나중에 요청올 결과를
미리 저장해두었다가 빠르게
서비스해 주는 것

6. Lots of
Data

7. Cache

8. Cache
Lots of
Data

9. CPU Cache

10. Browser Cache

11. Why is Cache?

13. Use Case: Login

14. Use Case: Login
Common Case

15. Use Case: Login
Common Case
Read From DB
Select * from Users where id=‘charsyam’;

16. 일반적인 DB구성
Master
Slave
REPLICATION/FailOver

17. 일반적인 DB구성
Master
Slave
REPLICATION/FailOver
모든 Traffic은 Master 가 처리 Slave는 장애 대비

18. 이러니깐 부하가!!!

19. 선택의 기로!!!

20. Scale UP
Vs
Scale OUT

21. Scale
UP

22. 초당 1000 TPS

23. 초당 3000 TPS
3배 처리 가능한 서버를 투입

24. Scale
OUT

25. 초당 1000 TPS

26. 초당 2000 TPS

27. 초당 3000 TPS

28. Scale Out을 선택!!!

29. Scale Out을 선택!!!
돈만 많으면 Scale Up도 좋습니다.

30. Request 분석
읽기 70%, 쓰기 30%

31. 분석 결론
읽기 70%,
읽기를 분산하자!!!

32. One Write
Master
+
Multi Read Slave

33. Client
Master
Slave
ONLY
WRITE
Slave Slave
Only READ
REPLICATION

34. Eventual
Consistency

35. Ex) Replication
Master
Slave
REPLICATION
Slave에 부하가 있거나, 다른 이유로, 복제가
느려질 수 있다. 그러나 언젠가는 같아진다.

36. Login 정보 중에 잠시
라도 서로 다르면 안되
는 경우는?

37. Login 정보 중에 잠시
라도 서로 다르면 안되
는 경우는?
=> 다시 처음으로!!!

38. 초반에는 행복했습니다.

39. Client
Master
Slave Slave Slave
REPLICATION
Slave Slave

40. 부하가 더 커지니!!!

41. Client
Master
Slave
REPLICATION
Slave Slave Slave Slave Slave
Slave Slave Slave Slave Slave Slave

42. 성능 향상이 미비함!!!
WHY?

43. 머신의 I/O는 Zero섬

46. Partitioning

47. Client
PART 1
Web Server
DBMS
PART 2
Web Server
DBMS
Scalable Partitioning

48. Paritioning
성능

49. Paritioning
성능
관리이슈

50. Paritioning
성능
관리이슈
비용

51. 간단한 해결책

52. DB 서버 Disk는 SSD
메모리도 데이터보다 많이

53. DB 서버 Disk는 SSD
메모리도 데이터보다 많이
=> 돈돈돈!!!

54. Why is Cache?

55. Use Case: Login

56. Use Case: Login
Read From Cache
Get charsyam

57. Use Case: Login
Apply Cache For Read

58. General Cache Layer
Cache
DBMS
Storage Layer
Application
Server
READ
WRITE
WRITE
UPDATE

59. Type 1: 1 Key – N Items
KEY
Profile:User_ID
Value
Profile
- LastLoginTime
- UserName
- Host
- Name

60. Type 2: 1 Key – 1 Item
KEY
name:User_ID
Value
UserName
LastLoginTime:User_ID LastLoginTime

61. Pros And Cons
Type 1: 1 Key – N Items
Pros: Just 1 get.

62. Pros And Cons
Type 1: 1 Key – N Items
Pros: Just 1 get.
Cons: if 1 item is
changed. Need Cache
Update
And Race Condition

63. Pros And Cons
Type 2: 1 Key – 1 Item
Pros: if 1 item is
changed, just change
that item.

64. Pros And Cons
Type 2: 1 Key – 1 Item
Pros: if 1 item is
changed, just change
that item.
Cons: Some Items can
be removed

65. 변화하지 않는 데이터
변화하는 데이터

66. 변화하지 않는 데이터
변화하는 데이터
Divide

67. Don’t Try Update After didn’t Read DB
Value
Profile
- LastLoginTime
- UserName
- Host
- Name

68. Don’t Try Update After didn’t Read DB
Value
Profile
- LastLoginTime
- UserName
- Host
- Name
EVENT: Update Only LastLoginTime

69. Don’t Try Update After didn’t Read DB
Value
Profile
- LastLoginTime
- UserName
- Host
- Name
EVENT: Update Only LastLoginTime
Read Cache: User Profile

70. Don’t Try Update After didn’t Read DB
Value
Profile
- LastLoginTime
- UserName
- Host
- Name
EVENT: Update Only LastLoginTime
Read Cache: User Profile
Update Data & DB
Just Save Cache

71. Don’t Try Update After didn’t Read DB
Value
Profile
- LastLoginTime
- UserName
- Host
- Name
EVENT: Update Only LastLoginTime
Read Cache: User Profile
Update Data & DB
Just Save Cache
It Makes Race Condition

72. Need
Global Lock

73. Need
Global Lock
=> 오늘은 PASS!!!

74. How to Test

75. Using Real Data
In 25 million User ID
100,000 Request
Reproduct From Log

76. Test Result
136 vs 1613
seconds
Memcache VS Mysql

77. Result of
Applying Cache

78. Select Query

79. CPU utility

80. 균등한 속도!!!
부하 감소!!!

81. What is Hard?

82. Keyand Value – SYNC Easy
1. Update To DB
2. Fail To Transaction
Fail!!!

83. Keyand Value – SYNC HARD
1. Update To DB
2. Update to CacheFail!!!

84. Keyand Value – How
1. Update To DB
2. Update to CacheFail!!!
RETRY BATCH DELETE

85. Data!
The most important thing

86. If data is not important
Cache Updating is not important
Login Count
Last Login Time
ETC

87. BUT!

88. If data is important
Cache Updating is important
Server Address
Data Path
ETC

89. HOW!

90. RETRY!
Retry, Retry, Retry
Solve Over 9x%.
Or Delete Cache!!!

91. Batch!
Queuing Service

92. Error Log
Queue
Error Log
Error Log
Error Log
Error Log
Batch
Processor
Cache
Server

93. Error Log
Queue
Error Log
Error Log
Error Log
Error Log
Batch
Processor
Cache
Server

94. Error Log
Queue
Error Log
Error Log
Error Log
Error Log
Batch
Processor
Cache
Server
UPDATE

95. Caches
Memcache
Redis

96. Memcache
Atomic Operation

97. Memcache
Atomic Operation
Key:Value

98. Memcache
Atomic Operation
Key:Value
Single Thread

99. Memcache
Over 100,000 TPS
Processing

100. Memcache
Max size of item
1 MB

101. Memcache
LRU,
ExpireTime

102. Redis
Key:Value

103. Redis
ExpireTime
Replication
Snapshot

104. Redis
Key:Value
Collection
List Sorted SetHash

105. 주의 사항

106. Item Size
1~XXX KB,
Item 사이즈는 적을수록
좋다.

107. Cache
In
Global Service

108. Memcached In Facebook
• Facebook and Google and Many Companies
• Facebook
– 하루 Login 5억명(한달에 8억명)(최신, 밑에는 작년 2010/04자료)
– 활성 사용자 7,000만
– 사용자 증가 비율 4일에 100만명
– Web 서버 10,000 대, Web Request 초당 2000만번
– Memcached 서버 805대 -> 15TB, HitRate: 95%
– Mysq server 1,800 대 Master/Slave(각각, 900대)
• Mem: 25TB, SQL Query 초당 50만번

109. Cache In Twitter(Old)
API WEB
DB DB
Page Cache

110. Cache In Twitter(NEW)
API WEB
DB DB DB
Page Cache
Fragment Cache
Row Cache
Vector Cache

111. Redis In Wonga
Using Redis for DataStore
Write/Read

112. Redis In Wonga
Flash Client
Ruby Backend

113. NoCache In Wonga
1 million daily Users
200 million daily HTTP Requests

114. NoCache In Wonga
1 million daily Users
200 million daily HTTP Requests
100,000 DB Operation Per Sec
40,000 DB update Per Sec

115. NoCache In Wonga

116. First Scale Out

117. First Setting – 3 Month

118. More Traffic

119. MySQL hiccups – DB Problems Began

120. Analysis About DBMS
ActiveRecord’s status Check caused
20% extra DB
60% of All Updates were done
on ‘tiles’ table

121. Applying Sharding 2xD

122. Result of Applying Sharding 2xD

123. Doubling MySQL

124. Result of Doubling MySQL

125. 50,000 TPS on EC2

126. Wonga Choose Redis!
But some failure
=> 오늘은 PASS!!!

128. Result!!!

129. Consistent Hashing

130. Proxy
Server
Server
Server
Server
Server
User Request
K = 10000 N = 5
Origin

131. Proxy
Server
Server
Server
Server
Server
User Request
K = 10000 N = 4
FAIL : Redistribution about 2000 Users

132. Proxy
Server
Server
Server
Server
Server
User Request
K = 10000 N = 5
RECOVER: Redistribution about 2500 Users

133. A
Add A,B,C Server

134. A
B
Add A,B,C Server

135. A
B
C
Add A,B,C Server

136. A
B
C
1
Add Item 1

137. A
B
C
1
2
Add Item 2

138. A
B
C
1
2
3
4
5
Add Item 3,4,5

139. A
B
C
2
3
4
5
Fail!! B Server

140. A
B
C
1
2
3
4
5
Add Item 1 Again -> Allocated C Server

141. A
B
C
1
2
3
4
5
1
Recover B Server -> Add Item 1

142. Thank You!

143. A
B
C
Real Implementation
A+1
A+2
A+3B+1
B+2
C+1
C+2
A+4
C+3
B+3