A talk covering some of the amazon AWS products for a Tuesday boston.rb meetup.

1. Cloud Computing featuring Amazon Web Services an unofficial presentation

2. I am... Brian Kaney from Vermonster . We are a 3 person shop in the Back Bay. We've been around for over 8 years and mostly do ruby, perl and python stuff. I do not work for Amazon. I don't know every Amazon offering – maybe you can help ?

3. Warning! There are lots of acronyms and abbreviations. I'll be bouncing from CLI, Web management console(s) and ruby code (and slides) This has been (somewhat) rehearsed but things happen... but at least you didn't have to pay and there's beer afterward.

5. Elastic Fundamentals

6. LoadBalance/AutoScale

7. Map Reduce

8. Monitoring

9. Other Services

10. Deploy / SCM ( System Configuration Management )

11. Architecture Thoughts

12. Closing

14. Elastic Fundamentals

15. LoadBalance/AutoScale

16. Map Reduce

17. Monitoring

18. Other Services

19. Deploy / SCM ( System Configuration Management )

20. Architecture Thoughts

21. Closing

22. Intro: Why Cloud?

23. Intro: Why Cloud?

24. Intro: Why Cloud?

25. Intro: Why Cloud?

26. Intro: Why Cloud?

27. Intro: Why Cloud?

28. Intro: Why Cloud?

29. Intro: Why Cloud? Temporary Events Unpredictability! Scalability Low Barrier to Entry Redundancy Batch Process

30. Intro: Uncontroversial Prediction Cloud computing is no longer vaporware and will continue growth .

32. Financial (MS Windows)

33. Biotech / Modeling / Scientific Community

34. Testing (devver, sauce labs)

35. Who else here clouds?

37. Rightscale (Amazon and others)

38. Rackspace

39. GoGrid

40. Google

41. Amazon

42. Eucalyptus (DIY)

44. Rightscale (Amazon and others)

45. Rackspace

46. GoGrid

47. Google

48. Amazon

49. Eucalyptus (DIY)

51. The most fully featured

52. The most mature API

53. And the only one I've really used

55. Min $0.10/hr ($70/month/instance)

56. Max $0.80/hr ($560/month/instance)

57. c1.medium $150-$200/month/instance

58. Reservations 1/3 the hourly price ($200 - $2000 per year)

59. NOTE: See aws.amazon.com for official pricing

61. Elastic Fundamentals

62. LoadBalance/AutoScale

63. Map Reduce

64. Monitoring

65. Other Services

66. Deploy / SCM ( System Configuration Management )

67. Architecture Thoughts

68. Closing

72. Instance API via link-local address

74. EC2

75. RightScale

76. Tons of others (http://github.com/search?type=Repositories&language=rb&q=amazon)

78. EC2

79. RightScale

80. Tons of others (http://github.com/search?type=Repositories&language=rb&q=amazon)

82. Virtualized Xen Servers

83. 6 sizes – c1.medium best bang for buck

84. Security Groups

85. Availability Zones ( different for everyone )

86. Launch from AMIs ( Amazon Machine Images )

87. Fundamentals: Launch an Instance!

89. Ubuntu.com ( http://help.ubuntu.com/community/EC2StartersGuide )

90. Well-maintained Ubuntu ones at ( http://alestic.com )

91. Consider re-bundle as part of deployment

92. Fundamentals: Let's bundle !

94. Buckets and blobs

95. AMIs live there

96. DNS mapping is cool

97. Pretty cheap and widely used – early offering

99. Network-attached drives

100. I use XFS

101. Snapshotting

102. Shared NEW

103. Fundamentals: Attach an EBS

105. Written in Erlang (rumors?)

106. “Offline” M/DB open-source clone (http://www.mgateway.com/mdb.html)

107. I haven't used SDB... anyone using it?

109. Not AMPQ

110. v2 has message locking ( recommended )

111. Order is not FIFO – but you can provide sequence metadata

112. Up to 8k in size

113. Fundamentals: Create SQS Messages

115. Elastic Fundamentals

116. LoadBalance/AutoScale (ELB/AS)

117. Map Reduce

118. Monitoring

119. Other Services

120. Deploy / SCM ( System Configuration Management )

121. Architecture Thoughts

122. Closing

123. ELB/AS: A Proposal I have a random idea: Let's launch an automatically-scalable , load-balanced cloud running a trivial sinatra application!

125. Register Instances

126. Balanced across all availability zones

127. Create a real CNAME in real life

129. Launch Configuration for each Group

131. Breach Duration – how long the breach is affected

132. ELB/AS: AutoScaling Concepts Trigger 1 Trigger 2 Trigger 3 Launch Config

133. ELB/AS: Launch Time!

135. Elastic Fundamentals

136. LoadBalance/AutoScale

137. Map Reduce

138. Monitoring

139. Other Services

140. Deploy / SCM ( System Configuration Management )

141. Architecture Thoughts

142. Closing

144. Functional Programming

145. For big data sets

146. Highly Parallelizable

147. Apache Hadoop

148. Uses S3 (HDFS?)

150. Then reduce combines based on keys

153. Key - value separated by “:”

154. Output of map and reduce should be similar.

155. JAR, Streaming , Pig or Hive

156. MapReduce: EMR Proposal I have another completely random idea: Let's process a data.gov feed of earthquake data to see the most seismically volatile regions using Elastic Map Reduce !

158. I don't have time for that

159. Let's look at some monitoring in the meantime.

161. Elastic Fundamentals

162. LoadBalance/AutoScale

163. Map Reduce

164. Monitoring

165. Other Services

166. Deploy / SCM ( System Configuration Management )

167. Architecture Thoughts

168. Closing

170. API provides aggregation ( sweet! )

171. Amazon has a nice UI

172. But account management is crappy

173. We can write some code for that!

175. Then use some HTML5 canvas action Thanks to a jQuery visualize plugin by Filament!

176. Our Map-Reduce Should be Done!

178. Elastic Fundamentals

179. LoadBalance/AutoScale

180. Monitoring

181. Map Reduce

182. Other Services

183. Deploy / SCM ( System Configuration Management )

184. Architecture Thoughts

185. Closing

187. Mechanical Turk

188. VPN NEW

189. Payment System (DevPay, Flexible Payments)

190. Load up a truck and physically ship your data to Amazon

191. Anyone using these?

193. Elastic Fundamentals

194. LoadBalance/AutoScale

195. Map Reduce

196. Monitoring

197. Other Services

198. Deploy / SCM ( System Configuration Management )

199. Architecture Thoughts

200. Closing

202. Capistrano

203. EC2-on-Rails (shouldn't it be rails-on-ec2?)

204. Pool-Party

205. Chef

206. Puppet

207. Rubber

208. (many others)

210. But direct ssh does not scale (try deploying to 100, 500 or 1,000 instances)

211. Vertebra (dead? uses XMPP)

212. Build one on a messaging queue? Use Nanite?

214. Elastic Fundamentals

215. LoadBalance/AutoScale

216. Map Reduce

217. Monitoring

218. Other Services

219. Deploy / SCM ( System Configuration Management )

220. Architecture Thoughts

221. Closing

223. Ephemeral Storage – /mnt

224. Messaging instead of forking/backgrounding (AMPQ)

225. Event Machine

226. Nanite (http://rubymanor.org/videos/nanite/)

228. (DB, Proxy, configuration/settings)

229. But we can if we drink the kool-aid! SQS S3 SDB LoadBalancer EBS

231. Predetermined technology ( SBD.. I'd rather have Postgres )

232. But we are agile, so less of an issue.

233. I can't back this up statistically, but my guess is most are using a hybrid architecture.

237. Log files ( Shared EBS helps )

239. Engine Yard

240. Right Scale

241. (others - lmgtfy)

242. Your Friendly Neighborhood Consultants

244. IRC – bkaney

245. Vermonster – http://www. vermonster.com