This is my delivery for assignment 2 in INN530 - Online Information Systems at QUT, created by Simen Fivelstad Smaaberg

1. Big data and cloud computing for
management
By
Simen Fivelstad Smaaberg (n8661260)

2. Abstract
 Big data
 What is it?
 What possibilities and challenges lays within?
 Cloud computing
 Software as a Service
 Platform as a Service
 Infrastructure as a Service
 How does big data relate to cloud computing?
 Google / Facebook
 How does Google and Facebook handle big data?
 What big data/cloud services do they offer for others to
use?

3. Background
 Internet today: a social web
 Huge amounts of data are created by users daily
 Creates your digital footprint
 Need for methods to handle all these data
 The data is used for analysis purposes
 Improve customer experiences
 Increase revenues

4. Big Data
 The vast volume of data in existence (Arthur, E
2013)
 Tweets, likes, videos, images, comments and so on
 Group of collected information, spans 3 V’s of data
management (Gartner 2011)
Figure 1: Big data 3V’s ( Datameer 2013)

5. Volume
 Big data is Big
 Exists in one size: large
 Challenges
 Physical storage space
 Logical structuring
 Scaling

6. Volume
 Big data is Big
 Exists in one size: large
 Challenges
 Physical storage space
 Logical structuring
 Scaling
 Opportunities
 Finding trends, patterns and relationships between data
 Possibilities for in depth analysis

7. Variety
 Challenges
 Infrastructure to handle
different kinds of media is
required.
 Challenging for
engineers
Figure 2: Big Data (Orange 2011)

8. Variety
 Challenges
 Infrastructure to handle
different kinds of media is
required.
 Challenging for
engineers
 Opportunities
 Finding patterns and
relationships between
different types of data
Figure 2: Big Data (Orange 2011)

9. Velocity
 Big data can have different kinds of time-sensitivity
 Real time vs non real time
 Challenges
 Infrastructure that can handle different kinds of time-
sensitivity

10. Velocity
 Big data can have different kinds of time-sensitivity
 Real time vs non real time
 Challenges
 Infrastructure that can handle different kinds of time-
sensitivity
 Opportunities
 Combine slow moving data with fast moving time
constrained data to give the user a better user experience

11. Big data success story: Santam Insurance
 About
 South Africas largest short-term insurance company
 Problem
 6-10% of premium revenue were fraud
 Solution
 Big Data prediction analysis

12. Big data success story: Santam Insurance
 About
 South Africas largest short-term insurance company
 Problem
 6-10% of premium revenue were fraud
 Solution
 Big Data prediction analysis
 Result
 First four months: 1.98 million USD saved
 First three years: ROI of 244%
 Insurance fraud syndicate disceovered

13. « Big data has the potential to change the way
governments, organizations, and academic
institutions conduct business and make
discoveries, and its likely to change how everyone
lives their day-to-day lives »
– Susan Hauser, VP Microsoft Enterprise and partner
group (Microsoft Enterprise team 2013)

14. Cloud computing
 Running applications elsewhere and accessing them through your
computer
 You get an “infinite” amount of storage space and computing power
 You pay for what you use
Figure 3: Cloud computing

15. Big Data and Cloud Computing
 Big data requires enormous amounts of storage
space
 Costly to build and maintain
 Huge engineering challenges

16. Big Data and Cloud Computing
 Big data requires enormous amounts of storage
space
 Costly to build and maintain
 Huge engineering challenges
 Solution: Cloud Computing!
 Put your data and programs into “the cloud”
 Avoid the hardware problem of big data
 Pay for the computing power you actually use,
 Opens up for smaller companies stepping into big data analysis

17. Big Data and Cloud Computing
 Big data requires enormous amounts of storage
space
 Costly to build and maintain
 Huge engineering challenges
 Solution: Cloud Computing!
 Put your data and programs into “the cloud”
 Avoid the hardware problem of big data
 Pay for the computing power you actually use,
 Opens up for smaller companies stepping into big data analysis
 Issues: Privacy and trust
 You put your data into someone else's hand
 Is that someone trustworthy?

18. Google and Hadoop
 2004: Google revolutionized the field of Big Data
and Cloud Computing
 Released papers describing how they handled these
topics

19. Google and Hadoop
 2004: Google revolutionized the field of Big Data
and Cloud Computing
 Released papers describing how they handled these
topics
 From this Yahoo spawned Hadoop
 Platform that can process and analyse huge amounts of
data on interconnected commodity servers
 Great fit for cloud computing
 Data is spread out and duplicated across servers
 Data is analysed in parallel through MapReduce
 Backbone of Twitter, Facebook, Yahoo and eBay

20. Google
 Huge competitor in the field of big data and cloud
computing
 Big data used internally
 Index searches
 Provide email services
 Provide advertizing
 External Big data and cloud services
 Software as a Service: Google docs, Gmail etc
 Platform as a Service: Google app engine
 Infrastructure as a Service: Google compute engine
 Gives developers access to the same infrastructure google itself
is run on

21. Facebook
 Forerunner in the field of Big Data
 Handling massive amounts of data daily
 2.5 billion status updates, wall posts, photos, videos and
comments
 2.7 billion likes
 300 million uploaded photos
 500 Tb new integrated data every day (2012)

22. Facebook
 Forerunner in the field of Big Data
 Handling massive amounts of data daily
 2.5 billion status updates, wall posts, photos, videos and
comments
 2.7 billion likes
 300 million uploaded photos
 500 Tb new integrated data every day (2012)
 Facebook is run on top of Hadoop
 100 Petabytes of storage
 Underpins analysis and everyday services
 New data is put into one of their Hadoop clusters physically
residing in one of their data centers
 Data is analysed when needed or at specific intervals
(hourly/daily) through MapReduce

23. Facebook Insight
 Tool that provides page owners (both facebook pages and
ordinary web pages) with metrics about their content.
(Facebook 2013)
 Number of visits
 Facebook referrals
 Visitor demographics (age, gender, location, language)
 Connects Facebook’s big data with users visiting your page to
provide metrics that can be used to improve your business’s
online performance
 Generated through Facebook’s Hadoop cluster
Figure 4: User demographics (Campalyst 2012)

24. Facebook graph search
 Search engine for your social circle (Bea, F 2013)
 Allows for search on relationships between
people, likes, comments, photos etc.
 Possible because of
Facebook’s big data
 Privacy concerns
Figure 5: Facebook restaurant search Figure 6: Facebook TV show search

25. The Future
 Big data and cloud computing is in constant change
 More and more usage areas are found
 Medical diagnostics
 Weather forecasts
 Particle physics
 Fraud prevention and detection
 Etc.
 Prediction: We have barely seen the start of its
dominance

26. References
 Arthur, E. 2013. "Big Data“. Alaska Business Monthly, vol. 29, no. 1, pp. 72-72. Retrieved from
http://search.proquest.com.ezp01.library.qut.edu.au/docview/1271622055
 Gartner. 2011. “Solving Big Data Challenge involves more than just managing volumes of data”.
Accessed June 4, 2013. http://www.gartner.com/newsroom/id/1731916
 Strickland, J. “How Cloud Computing Works”. Accessed June 6, 2013.
http://computer.howstuffworks.com/cloud-computing/cloud-computing.htm
 Gartner. “Software as a Service (SaaS)”. Accessed June 6. 2013. http://www.gartner.com/it-
glossary/software-as-a-service-saas/
 Chong, R. 2011. “The perfect marriage: Hadoop and Cloud”. Accessed June 4, 2013.
http://thoughtsoncloud.com/index.php/2011/10/the-perfect-marriage-hadoop-and-cloud/
 Microsoft Enterprise Team. “The Big Bang: How the Big Data Explosion Is Changing the
World”. Last Modified March 27, 2013.
http://www.microsoft.com/enterprise/it-trends/big-data/articles/The-Big-Bang-How-the-Big-Data-
Explosion-Is-Changing-the-World.aspx#fbid=8RIFw1BLCG2
 Metz, C. 2011. “How Yahoo Spawner Hadoop, the Future of Big Data”. Accessed June 5, 2013.
http://www.wired.com/wiredenterprise/2011/10/how-yahoo-spawned-hadoop/all/1
 Big Data Insights. 2013. “How Facebook uses Hadoop and Hive”. Accessed June 05, 2013.
http://hortonworks.com/blog/how-facebook-uses-hadoop-and-hive/
 Facebook. “Insights”. Last modified May 30, 2013.
https://developers.facebook.com/docs/insights/
 Bea, F. 2013. “How Facebook’s Graph Search Works…Sort Of". Accessed June 05, 2013.
http://www.digitaltrends.com/social-media/how-facebook-graph-search-works/
 IBM. 2013. “IBM Business Analytics SPSS: Santam insurance”. Last modified May 28, 2013.
http://www-01.ibm.com/software/success/cssdb.nsf/CS/SANS-
985HX2?OpenDocument&Site=default&cty=en_us

27. References - Illustrations
 Datameer. 2013. “What is Big Data?”. Digital
Image. Viewed June 8, 2013.
http://www.datameer.com/product/big-data.html
 Orange. 2011. “Analyst insight”. Digital Image.
Viewed June 11, 2013. http://www.orange-
business.com/en/magazine/analyst-insight-
december-2011
 Campalyst. 2012. “How to measure website visitors’
demographics: hidden Facebook Insights gem”.
Digital Image. Viewed June 11, 2013.
http://blog.campalyst.com/2012/10/10/how-to-
measure-website-visitors-demographics-hidden-
facebook-insights-gem/