Palestra na Algar Telecom - Tech4All - Setembro 2015

1. Desmistificando
NoSQL e Novas
Tecnologias de
Bancos de Dados
Fabíola
S.
F.
Pereira
Algar
Telecom
Set
2015

2. P roblema: “Crawling” T witter
t

3. P roblema: “Crawling” T witter
t

4. P roblema: “Crawling” T witter
t

5. P roblema: “Crawling” T witter
t

6. P roblema: “Crawling” T witter
t
C o m o o bte r o m o m e nt o e m q u e as a r e s t as a p a r e c e m
e d e s a p a r e c e m n a r e d e ?

7. P roblema: “Crawling” T witter
t
C o m o o bte r o m o m e nt o e m q u e as a r e s t as a p a r e c e m
e d e s a p a r e c e m n a r e d e ?
2 3 0 M d e u s u á ri o s e 7 0 0 M d e a r e s t as

8. P roblema: “Crawling” T witter
2 3 0 M d e u s u á ri o s e 7 0 0 M d e a r e s t as
C o m o o bte r o m o m e nt o e m q u e as a r e s t as a p a r e c e m
e d e s a p a r e c e m n a r e d e ?

9. P roblema: “Crawling” T witter
2 3 0 M d e u s u á ri o s e 7 0 0 M d e a r e s t as
C o m o o bte r o m o m e nt o e m q u e as a r e s t as a p a r e c e m
e d e s a p a r e c e m n a r e d e ?

10. P roblema: “Crawling” T witter
2 3 0 M d e u s u á ri o s e 7 0 0 M d e a r e s t as
C o m o o bte r o m o m e nt o e m q u e as a r e s t as a p a r e c e m
e d e s a p a r e c e m n a r e d e ?
u1,
u2,
12/04,
13/06
u1,
u3,
01/01,
02/01
u4,
u5,
09/11,
25/12
…

11. P roblema: “Crawling” T witter
2 3 0 M d e u s u á ri o s e 7 0 0 M d e a r e s t as
C o m o o bte r o m o m e nt o e m q u e as a r e s t as a p a r e c e m
e d e s a p a r e c e m n a r e d e ?
u1,
u2,
12/04,
13/06
u1,
u3,
01/01,
02/01
u4,
u5,
09/11,
25/12
…

12. P roblema: Futebol

13. P roblema: Futebol
C o m o p ro c e s s a r ev e nt o s e m te mp o r e a l ?

14. P roblema: Futebol
C o m o p ro c e s s a r ev e nt o s e m te mp o r e a l ?
D a d o s n ã o s ã o a rma z e n a d o s !

15. P roblema: E-commerce
C o m o vo c ê c o n s t r u i ri a u m s i te ma e - c o m m e r c e s e m
j o i n s e s e m n o rma l i z a ç ã o ?

16. P roblema: E-commerce
C o m o vo c ê c o n s t r u i ri a u m s i te ma e - c o m m e r c e s e m
j o i n s e s e m n o rma l i z a ç ã o ?
E s c a l á v e l

17. História
Modelo de Dados
Consistência e NoSQL
Usando Bancos NoSQL
Exemplos e mais exemplos
Referências legais

18. Ascenção
Relacional
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

19. Persistência
Integração
SQL
Transações
Relatórios
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

20. Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

21. Bancos orientados a
objetos
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

22. Dominância
Relacional
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

23. Billing
Inventário
Catálogo
Relatórios
Mecanismo de
Integração
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

24. Dominância
Relacional
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

25. Grande Tráfego de
Dados
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

26. Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

27. Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

28. SQL
SQL
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

29. Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

30. Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

31. NoSQL
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

32. Definição de NoSQL
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

33. Características de
NoSQL
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

34. Não-relacional
Open-source
cluster-friendly
século XXI - Web
schema-less
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

35. História
Modelo de Dados
Consistência e NoSQL
Usando Bancos NoSQL
Exemplos e mais exemplos
Referências legais

36. Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

37. Documento
Grafo
Key-value
Column-
family
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

38. Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

39. Documento
schema-less
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

40. Documento
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

41. Documento
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

42. DocumentoKey-value
Aggregate-oriented
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

43. Aggregate = documentoAggregate = value
Documento
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

44. Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

45. DocumentoKey-value
Aggregate-oriented
Column-family
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

46. Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

47. Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

48. Grafo
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

49. Grafo
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

50. Grafo
Column-family
Documento
Key-value

51. Grafo
Column-family
Documento
Key-value
Aggregate

52. História
Modelo de Dados
Consistência e NoSQL
Usando Bancos NoSQL
Exemplos e mais exemplos
Referências legais

53. SQL = ACID
NoSQL = BASE
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

54. Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

55. Grafo
ACID
Column-family
Documento
Key-value
Aggregate

56. Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

57. Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

58. Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

59. Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

60. Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

61. Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

62. Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

63. Consistência
Disponibilidade
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

64. Teorema CAP
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

65. Teorema CAP
Consistency
Availability
Partition tolerance
Escolha 2
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

66. AvailabilityConsistency
Partition tolerance
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

67. Consistência
Disponibilidade
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

68. Consistência
Tempo de
Resposta
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

69. Shopping
Shopping
Shopping
Shopping
ShoppingShopping
Shopping Shopping

70. História
Modelo de Dados
Consistência e NoSQL
Usando Bancos NoSQL
Exemplos e mais exemplos
Referências legais

71. Definição de NoSQL

72. Definição de NoSQL
Desenvolvimento
fácil
Dados em larga
escala
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

73. Billing
Inventário
Catálogo
Relatórios
Mecanismo de
Integração
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

74. Billing
Inventário
Catálogo
Relatórios
Bancos de
aplicações
WS, ESB, …
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

75. NoSQL?
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

76. Persistência
Poliglota
Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

77. Copied
from:
Introduc<on
to
NoSQL.
Mar<n
Fowler's
talk
from
the
GOTO
Aarhus
Conference
2012.
hOps://www.youtube.com/watch?v=qI_g07C_Q5I

80. História
Modelo de Dados
Consistência e NoSQL
Usando Bancos NoSQl
Exemplos e mais exemplos
Referências legais

81. P roblema: “Crawling” T witter
t
C o m o o bte r o m o m e nt o e m as a r e s t as a p a r e c e m e
d e s a p a r e c e m n a r e d e ?
2 3 0 M d e u s u á ri o s e 7 0 0 M d e a r e s t as

82. P roblema: “Crawling” T witter
2 3 0 M d e u s u á ri o s e 7 0 0 M d e a r e s t as
u1,
u2,
12/04,
13/06
u1,
u3,
01/01,
02/01
u4,
u5,
09/11,
25/12
…
C o m o o bte r o m o m e nt o e m as a r e s t as a p a r e c e m e
d e s a p a r e c e m n a r e d e ?

83. P roblema: “Crawling” T witter
u1,
u2,
12/04,
13/06
u1,
u3,
01/01,
02/01
u4,
u5,
09/11,
25/12
…
Volume de
dados
Map Reduce

84. P roblema: Futebol
C o m o p ro c e s s a r ev e nt o s e m te mp o r e a l ?
D a d o s n ã o s ã o a rma z e n a d o s !

85. P roblema: Futebol
C o m o p ro c e s s a r ev e nt o s e m te mp o r e a l ?
D a d o s n ã o s ã o a rma z e n a d o s !

86. P roblema: Recomendação
C o m o f a z e r r e c o m e n d a ç õ e s a p a rt i r d o h i s t ó ri c o d o
u s u á ri o , d e s u a r e d e s o c i a l , d e s u as av a l i a ç õ e s , … ?

87. P roblema: Recomendação
C o m o f a z e r r e c o m e n d a ç õ e s a p a rt i r d o h i s t ó ri c o d o
u s u á ri o , d e s u a r e d e s o c i a l , d e s u as av a l i a ç õ e s , … ?
C o mp le x i d a d e d o r e l ac i o n a m e nt o e nt r e o s d a d o s

88. P roblema: Recomendação
C o m o f a z e r r e c o m e n d a ç õ e s a p a rt i r d o h i s t ó ri c o d o
u s u á ri o , d e s u a r e d e s o c i a l , d e s u as av a l i a ç õ e s , … ?
C o mp le x i d a d e d o r e l ac i o n a m e nt o e nt r e o s d a d o s

89. P roblema: Recomendação

90. P roblema: E-commerce
C o m o vo c ê c o n s t r u i ri a u m s i te ma e - c o m m e r c e s e m
j o i n s e s e m n o rma l i z a ç ã o ?
E s c a l á v e l

91. P roblema: E-commerce
ü U s ar agre gaç ão e an in hame nt o ao invés
d e j oi n s
ü D up li c ar d ad os ao invés d e n ormali z aç ão
ü C on s i s t ê nc i a eve nt ual
E s c a l á v e l

92. P roblema: E-commerce
E s c a l á v e l

93. História
Modelo de Dados
Consistência e NoSQL
Usando Bancos NoSQl
Exemplos e mais exemplos
Referências legais

96. Desmistificando
NoSQL e Novas
Tecnologias de
Bancos de Dados
Fabíola
S.
F.
Pereira
Algar
Telecom
Set
2015