Big Data overview, use cases, technology and opportunities. Presented at Everis on the 11th of April 2013

1. Big Data
Wilson Lucas
wilsonlucas@yahoo.com
©Wilson Lucas 2013

2. Apresentação
Wilson Lucas
wilsonlucas@yahoo.com
©Wilson Lucas 2013

3. O que é o “Big Data”
"Big data" is
• high-volume,
• high-velocity and
• high-variety
information assets that demand
cost-effective, innovative forms of
information processing for
enhanced insight and decision
making.
 An enhanced insight that leads to improved decision making and value
creation.
Source: The Importance of 'Big Data': A Definition, Mark Beyer, Douglas Laney, G00235055
©Wilson Lucas 2013

4. Big Data
“80% of data is now unstructured content such as email, videos,
and other user-generated content which must be combined
with structured data to produce the information businesses
need to serve consumers individually.”
Weblogs, social media, server logs, sensores, emails, fotografias.
O crescimento da electrónica de consumo, a baixa de preços continuada
de processadores, armazenamento e comunicações.
Transformação : o potencial destes dados pode agora ser explorado
pelas organizações.
 Aumento do valor estratégico da informação.
“Infonomics is recent term to describe the study and emergent discipline of
quantifying, managing and leveraging information as a formal business asset.”
wikipedia
©Wilson Lucas 2013

5. Big Data
• Volume
Crescimento rápido de dados estruturados, não estruturados, internos e
externos. Ex: Um motor a jacto produz cerca de 10TB de dados em 30 minutos...
• Velocidade
Necessidades de negócio não antecipadas.
Ex: Fluxo de opiniões e relações geradas,
alta velocidade e frequência dos tweets
• Variedade
Vários formatos, diferindo dos tradicionais com taxas de mudança bastante altas.
Novos tipos de dados serão necessários para capturar a informação resultante.
Ex: novos sensores implementados, novas campanhas de marketing executadas
• Valor
Existe pouco valor per se nestes dados.
Ex: Se perdermos 5 minutos das transacções de um cliente numa webstore, não irá
influenciar a analise sobre o seu comportamento.
©Wilson Lucas 2013

6. Alguns cenários de utilização
©Wilson Lucas 2013

7. Alguns cenários de utilização
Desafio actual Novos dados Possibilidades
Cuidados de Saúde: Monitorização remota do Cuidados preventivos,
paciente hospitalização reduzida
deslocação e consultas
dispendiosas
Fabricação: Sensores de produto Diagnostico automático
Suporte presencial
Serviços com base em Dados de localização em Geo-advertising, tráfego,
localização geográfica tempo real busca local
Sector Publico: Inquéritos dos cidadãos Serviços à medida,
Serviços padrão redução de custos
Comercio e retalho: Social media Analise de sentimento e
Marketing “generalizado” segmentação
©Wilson Lucas 2013

8. Realidade vs. Ficção
©Wilson Lucas 2013

9. Tendências
©Wilson Lucas 2013

10. Tendências
Hype Cycle for emerging technologies

©Wilson Lucas 2013

11. Aplicado ao Marketing
©Wilson Lucas 2013

12. Questões que o Negócio quer ver resolvidas
Gartner
©Wilson Lucas 2013

13. Exemplo: Consumer Analytics
©Wilson Lucas 2013

14. Information Storage 2020
Mito da BD como única fonte de informação
…No DISK, No SQL, Very Cloudy
2011
• Content File Store
• DBMS • DBMS Cloud Services
• Column Store DBMS
• noSQL
• In-Memory DBMS
• Big Data Solutions
©Wilson Lucas 2013

15. Tecnologia
©Wilson Lucas 2013

16. O papel da tecnologia
• Hadoop project
It is a framework which allows processing of large data sets across cluster of
computers (commodity hardware).
Hadoop MapReduce Its a programming mode which supports parallel processing of
large datasets
http://gigaom.com/cloud/what-it-really-means-when-someone-says-hadoop/
©Wilson Lucas 2013

17. O papel da tecnologia
• Ecossistema Hadoop
http://tushar686.wordpress.com/2012/04/04/hadoop-and-mapreduce//
©Wilson Lucas 2013

18. Tecnologia
"How to Choose the Right Apache Hadoop Distribution" — Vendors offer
Apache Hadoop distributions with preintegrated projects, but different
vendors offer different combinations, at differing release levels.”
“Interest in using Hadoop to solve big data challenges has increased
significantly in the past 12 months. “
“IT architects, business leaders and data scientists involved in "big data"
projects can easily go wrong when they construct an Apache Hadoop
stack, because the 20 or more potential components ("projects") are not
integrated as commercial software packages are.”
©Wilson Lucas 2013

19. ©Wilson Lucas 2013

20. IBM Will Boost Analytics Ability by Buying Search Provider Vivisimo
Aster Data Purchase Shows Teradata's Vision Is Deeper Than 'Big Data'
The planned acquisition of Aster Data will give Teradata a distributed data warehouse that
supports unstructured content and more aspects of extreme data than competitors do now.
Netezza Acquisition Will Boost IBM Against Oracle, Teradata
Hadoop Distribution Seeks to Leverage Intel's Microprocessor Strengths
Acquire, organize, and analyze big data
Uncover hidden relationships that lead to new perspectives
Turn analytics into better decision-making
 É fundamental perceber o que os dados significam para a organização.
©Wilson Lucas 2013

21. Investimento
©Wilson Lucas 2013

22. Investimento
“Big Data Drives Rapid Changes in Infrastructure and $232 Billion in IT Spending
through 2016
Big data has become a major driver of IT spending. The benefits to organizations
for adding big data to their information management and analytics infrastructure
will force a more rapid cycle of replacing existing solutions.”
©Wilson Lucas 2013

23. Comparison of Data Characteristics by Industry
Media and Services
Manufacturing and
Natural Resources
Communications,
Wholesale Trade
Transportation
Banking and
Government
Healthcare
Education
Securities
Insurance
Providers
Utilities
Retail
Volume
of Data
Velocity
of Data
Variety
of Data
Potential big data opportunity on each dimension is:
Very hot (compared with other industries)
Hot
Moderate
Low
Very low (compared with other industries)
©Wilson Lucas 2013

24. Comparison o Spending Intensity by Industry
Media and Services
Manufacturing and
Natural Resources
Communications,
Wholesale Trade
Transportation
Banking and
Government
Healthcare
Education
Securities
Insurance
Providers
Utilities
Retail
Hardware
Software
Services
Potential big data opportunity on each dimension is:
Very hot (compared with other industries)
Hot
Moderate
Low
Very low (compared with other industries)
©Wilson Lucas 2013

25. Big Data
Obrigado!
Wilson Lucas
wilsonlucas@yahoo.com
©Wilson Lucas 2013