Distributed File System

1. CS 704DAdvanced Operating System( Distributed File System) Debasis Das

2. Distributed File System IT 703D Debasis Das

4. Software providing specific functionality

5. Server

6. An instance of service running on a specific machine

7. Client

8. Process that requests a service

9. Client Interface

10. A set of operations that can be done by the client

11. Inter-machine Interface

12. Interface for cross-machine interactionIT 703D Debasis Das

14. Yet the access by the user is the same as if the whole file system was locally available (transparency)

15. A component unit is the smallest set of files that can be stored on a nodeIT 703D Debasis Das

17. Management of storage service ( disk/block service)

18. True file service

19. Management of files, creation, deletion, sharing, accessing etc.

20. Naming service

21. Mapping between text names and actual files9directory service)IT 703D Debasis Das

23. User mobility

24. Performance

25. Simplicity and ease of use

26. Scalability

27. High Availability

28. High reliability

29. Data integrity

30. Security

31. HeterogeneityIT 703D Debasis Das

33. Just a collection of data in the unstructured form, structured files have the data structured in some form, a collection of records

34. structured files can be non-indexed or indexed

35. Most modern file systems are unstructured

36. Mutable & immutable files

37. The file is changed when any new write happens. In immutable model, a new version is createdIT 703D Debasis Das

39. Remote service model: data packing, communication overheads could be significant

40. Data caching model : bring data in a block and keep it cached, cache coherency & other cache problems exist

41. Unit of data transfer

42. File level transfer model: move the whole file

43. Block level transfer model: move block(s) of data

44. Byte level transfer model: move bytes of data

45. Record level transfer model: move records in case of a structured file with recordsIT 703D Debasis Das

47. Ache location

48. Modification propagation

49. Cache validationIT 703D Debasis Das

51. Easy to implement, transparent to clients, easy to share

52. Client’s disk

53. Does not work with diskless stations, reliable against crashes,scalable

54. Client’s main memory

55. Maximum performance gain, workstations can be diskless, scalableIT 703D Debasis Das

57. All writes are written through to server disk

58. Works best when there are more reads than writes

59. Delayed write

60. Write on ejection from cache

61. Periodic write

62. Write on closeIT 703D Debasis Das

64. Checking before every approach

65. Periodic checking

66. Check on file open

67. Server initiated approach

68. Server keeps track of when a file is opened and in what mode

69. Whenever there is a potential for conflict, server must act

70. Cannot let a file open for read to be opened for writeIT 703D Debasis Das

72. File servers will have to be Stateful, problems if server fails

73. Check on open, client initiated validation is still required. A client can cache a file, open and the close after use. On re-opening the cache must be validated againIT 703D Debasis Das

75. Differences with caching

76. Replica is associated with server

77. Replication is decided by availability and performance requirements

78. Replica is more persistent, widely known, secure, available, complete and accurate

79. Cache needs to be validated against a replicaIT 703D Debasis Das

81. Increased reliability

82. Improved response time

83. Reduced network traffic

84. Improved system throughput

85. Better scalability

86. Autonomous operation

87. Replicate all required files on the node that requires themIT 703D Debasis Das

89. All replicas of a immutable file can have the same name

90. Mapping system must identify the latest copy to kernel. Some distance information for replicas also required

91. Replication control

92. Explicit replication

93. Users control the replicas

94. Implicit/lazy replication

95. System takes care of how many replicas and where to be placed. Lazy replication is done when system s free to do itIT 703D Debasis Das

97. Immutable files only

98. Read any write all protocol

99. read from any copy, write to all copies, lock them first

100. Available copies protocol

101. Write all is difficult to implement if one or more server with copies are down. Update the available copies only

102. Primary copy protocol

103. Write to designated primary copy, update others

104. Quorum based protocols

105. n copies of a file exists, r read quorum, w write quorum such that r + w > n. at least one up to date copy existsIT 703D Debasis Das

107. A file may not be available to some nodes on failure & network partitioning

108. Replication increase availability

109. Robustness

110. Power to survive crashes, weakening of storage media

111. Recoverability

112. Roll back to an earlier consistent stateIT 703D Debasis Das

114. Non-volatile storage

115. Stable storage

116. Use of redundant units to ensure robustness

117. Use the same ordinary unitsIT 703D Debasis Das

119. Server maintains the state of the operations

120. Stateless server

121. Does not maintain state

122. Effect of stateless server on fault tolerance

123. Unlike Stateful servers no complex crash recover required

124. Client simply send the request again, no recovery process required

125. Identifier translation imposes a time penalty

126. To keep the operations indempotent, arguments need to be sent with every requestIT 703D Debasis Das

128. All or nothing

129. Properties

130. Atomicity

131. Failure atomicity

132. If fails, roll back to earlier state

133. Concurrency atomicity

134. On concurrent access, other processes cannot see intermediate states only the final result (consistency property)

135. Serializability

136. Done in some serial order

137. Permanence

138. Once done, it is permanentIT 703D Debasis Das

140. Failure can leave a file in inconsistent state

141. With transaction feature, the sate can be rolled back

142. Allowing concurrent sharing of mutable files

143. Multiple client operations if not executed in proper order can leave files inconsistent

145. Begin transaction

146. End transaction

147. Abort transactions

148. Some lower level primitives

149. T-read

151. Complete the transaction, else abort transaction

152. Commit if transaction completed successfully

153. File versions approach

155. Optimized locking

156. Type specific locking

157. Read and write locks. When a read lock is set, other reads may be allowed but not write; When a write lock is on other operations are permitted

158. Intention-to-write locking; along with commit lock permits better concurrency

159. Two phase locking

161. File, page or a record

162. Handling of locking deadlocks

163. Avoidance, detection, timeouts

164. Optimistic concurrency control

165. Let the first phase proceed without restrictions, check for validity before commitmentAdvantages Free from deadlocks, allows maximum parallelism

167. Cache whenever possible: Never know when you are going to need the item again, keep a local copy

168. Exploit usage properties: group files

169. Minimize system wide knowledge and change

170. Trust the fewest possible entities

171. Batch if possible