Sentiment analysis of online social networks, such as Twitter, is a rapidly growing research area with an increasing number of applications such as: informing disaster response; tracking spread of contagious disease; early forecast of civil unrest. This talk will present an approach for building scalable social media analysis platforms in Clojure

1. Sentiment Analysis of
the Twittersphere
@APassionForCode

4. Leiningen Versus the Ants!
Clojure Versus Java!
FP Versus OOP

5. Apache Ant
Leiningen Versus the Ants!
Clojure Versus Java!
FP Versus OOP

6. Apache Ant
Leiningen Versus the Ants
Clojure Versus Java!
FP Versus OOP

7. Apache Ant
Leiningen Versus the Ants
Clojure Versus Java
Functional Versus Object Oriented

8. “You wanted a banana but what you got was a
gorilla holding the banana and the entire jungle.”
Joe Armstrong - inventor of Erlang

9. “Sometimes, the elegant implementation is just
a function. Not a method. Not a class. Not a
framework. Just a function.”
John Carmack - Lead on Doom, Quake, Rage

11. Blood aches pain
sick shivers spasm
vomit dizzy fainting
colic and in a
coma!
1 FAVORITE

12. Blood aches pain
sick shivers spasm
vomit dizzy fainting
colic and in a
coma!
1 FAVORITE
Scores HIGH for flu symptoms

13. Blood aches pain
sick shivers spasm
vomit dizzy fainting
colic and in a
coma!
1 FAVORITE
Who is Catalina Rubottom?

14. Blood aches pain
sick shivers spasm
vomit dizzy fainting
colic and in a
coma!
1 FAVORITE
30 million geo-tagged
tweets sent from UK
HDFS/Hadoop
Mongo/Aggregation
Mongo/MapReduce
Postgres

15. How can we do fast, real time analytics of
social media?

16. Store t his!
Not this!

17. WARNING!
You’re about to see real Clojure code!

18. increment (1)

19. (increment 1)

20. increment (1)

21. (increment 1)
Co ng ratu lat io ns - yo u’re no w fu lly qualifie d!

22. ptaoussanis/carmine

23. (:require
[taoensso.carmine :as car])
!
!
(defn create-tweet-id []
(first
(wcar*
(car/incr "global:tweet.id"))))
!
!
(defn set-values [sentiment location created-at]
(let [tweed-id (create-tweet-id)]
(wcar*
(car/setbit sentiment tweet-id 1)
(car/setbit location tweet-id 1)
(car/setbit created-at tweet-id 1))))

24. (:require
[taoensso.carmine :as car])
!
!
(defn create-tweet-id []
(first
(wcar*
(car/incr "global:tweet.id"))))
!
!
(defn set-values [sentiment location created-at]
(let [tweed-id (create-tweet-id)]
(wcar*
(car/setbit sentiment tweet-id 1)
(car/setbit location tweet-id 1)
(car/setbit created-at tweet-id 1))))

25. (:require
[taoensso.carmine :as car])
!
!
(defn create-tweet-id []
(first
(wcar*
(car/incr "global:tweet.id"))))
!
!
(defn set-values [sentiment location created-at]
(let [tweed-id (create-tweet-id)]
(wcar*
(car/setbit sentiment tweet-id 1)
(car/setbit location tweet-id 1)
(car/setbit created-at tweet-id 1))))

26. (:require
[taoensso.carmine :as car])
!
!
(defn create-tweet-id []
(first
(wcar*
(car/incr "global:tweet.id"))))
!
!
(defn set-values [sentiment location created-at]
(let [tweed-id (create-tweet-id)]
(wcar*
(car/setbit sentiment tweet-id 1)
(car/setbit location tweet-id 1)
(car/setbit created-at tweet-id 1))))

27. (:require
[taoensso.carmine :as car])
!
!
(defn create-tweet-id []
(first
(wcar*
(car/incr "global:tweet.id"))))
!
!
(defn set-values [sentiment location created-at]
(let [tweed-id (create-tweet-id)]
(wcar*
(car/setbit sentiment tweet-id 1)
(car/setbit location tweet-id 1)
(car/setbit created-at tweet-id 1))))
find value for key
set bit to 1
find bit of index

28. (wcar*
(car/bitcount "SCOTLAND")
(car/bitcount "JOVIALITY")
(car/bitcount "26062014114532"))

29. ENG
SCT
WAL
NIR
key value

30. key tweet-id
(car/setbit “ENG” 0 1)
ENG 1
SCT 0
WAL 0
NIR 0
key value

31. key tweet-id
(car/setbit “SCT” 1 1)
ENG 0 1
SCT 1 0
WAL 0 0
NIR 0 0
key value

32. key tweet-id
(car/setbit “ENG” 2 1)
ENG 1 0 1
SCT 0 1 0
WAL 0 0 0
NIR 0 0 0
key value

33. key tweet-id
(car/setbit “SCT” 15 1)
ENG 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1
SCT 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0
WAL 0 1 0 1 0 0 0 0 0 1 0 0 0 0 0 0
NIR 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0
key value

34. (car/bitcount "ENG")
ENG 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1
SCT 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0
WAL 0 1 0 1 0 0 0 0 0 1 0 0 0 0 0 0
NIR 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0
8
key value

35. (car/bitcount "SCT")
ENG 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1
SCT 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0
WAL 0 1 0 1 0 0 0 0 0 1 0 0 0 0 0 0
NIR 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0
8
4
key value

36. (car/bitcount "WAL")
ENG 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1
SCT 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0
WAL 0 1 0 1 0 0 0 0 0 1 0 0 0 0 0 0
NIR 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0
8
4
3
key value

37. (car/bitcount "NIR")
ENG 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1
SCT 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0
WAL 0 1 0 1 0 0 0 0 0 1 0 0 0 0 0 0
NIR 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0
8
4
3
1
key value

38. JOVIAL
SHY
HOSTILE
FATIGUE
key value

39. key tweet-id
(car/setbit “JOVIAL” 0 1)
JOVIAL 1
SHY 0
HOSTILE 0
FATIGUE 0
key value

40. key tweet-id
(car/setbit “HOSTILE” 1 1)
JOVIAL 0 1
SHY 0 0
HOSTILE 1 0
FATIGUE 0 0
key value

41. key tweet-id
(car/setbit “SHY” 2 1)
JOVIAL 0 0 1
SHY 1 0 0
HOSTILE 0 1 0
FATIGUE 0 0 0
key value

42. key tweet-id
(car/setbit “JOVIAL” 15 1)
JOVIAL 1 0 1 0 1 1 1 0 0 1 0 1 0 0 0 1
SHY 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
HOSTILE 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0
FATIGUE 0 1 0 1 0 0 0 0 1 0 0 0 1 0 0 0
key value

43. (car/bitcount [*])
JOVIAL 1 0 1 0 1 1 1 0 0 1 0 1 0 0 0 1
SHY 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0
HOSTILE 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0
FATIGUE 0 1 0 1 0 0 0 0 1 0 0 0 1 0 0 0
8
1
3
4
key value

44. How many people in England are Happy
about the referendum result?
(wcar*
(car/bitop "AND" “ENGLAND&JOVIALITY" "ENGLAND" "JOVIALITY")
(car/expire “ENGLAND&JOVIALITY" 10)
(car/bitcount “ENGLAND&JOVIALITY"))

45. How many people in England are Happy
about the referendum result?
(wcar*
(car/bitop "AND" “ENGLAND&JOVIALITY" "ENGLAND" "JOVIALITY")
(car/expire “ENGLAND&JOVIALITY" 10)
(car/bitcount “ENGLAND&JOVIALITY"))

46. How many people in England are Happy
about the referendum result?
(wcar*
(car/bitop "AND" “ENGLAND&JOVIALITY" "ENGLAND" "JOVIALITY")
(car/expire “ENGLAND&JOVIALITY" 10)
(car/bitcount “ENGLAND&JOVIALITY"))

47. How many people in England are Happy
about the referendum result?
(wcar*
(car/bitop "AND" “ENGLAND&JOVIALITY" "ENGLAND" "JOVIALITY")
(car/expire “ENGLAND&JOVIALITY" 10)
(car/bitcount “ENGLAND&JOVIALITY"))

48. How many people in England are Happy
about the referendum result?

49. How many people in England are Happy
about the referendum result?
ENG 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1

50. How many people in England are Happy
about the referendum result?
ENG 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1
JOVIAL 1 0 1 0 1 1 1 0 0 1 0 1 0 0 0 1

51. How many people in England are Happy
about the referendum result?
ENG 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1
JOVIAL 1 0 1 0 1 1 1 0 0 1 0 1 0 0 0 1
1
AND
(car/bitop "AND" “ENGLAND&JOVIALITY" "ENGLAND" "JOVIALITY")

52. How many people in England are Happy
about the referendum result?
ENG 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1
JOVIAL 1 0 1 0 1 1 1 0 0 1 0 1 0 0 0 1
0 1
AND
(car/bitop "AND" “ENGLAND&JOVIALITY" "ENGLAND" "JOVIALITY")

53. How many people in England are Happy
about the referendum result?
ENG 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1
JOVIAL 1 0 1 0 1 1 1 0 0 1 0 1 0 0 0 1
0 0 1
AND
(car/bitop "AND" “ENGLAND&JOVIALITY" "ENGLAND" "JOVIALITY")

54. How many people in England are Happy
about the referendum result?
ENG 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1
JOVIAL 1 0 1 0 1 1 1 0 0 1 0 1 0 0 0 1
0 0 1 0 1 1 0 0 0 0 0 0 0 0 0 1
AND
(car/bitop "AND" “ENGLAND&JOVIALITY" "ENGLAND" "JOVIALITY")

55. ENG 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1
JOVIAL 1 0 1 0 1 1 1 0 0 1 0 1 0 0 0 1
0 0 1 0 1 1 0 0 0 0 0 0 0 0 0 1
(car/bitcount “ENGLAND&JOVIALITY")
4
How many people in England are Happy
about the referendum result?

56. How many people in Scotland are Tired
and Grumpy after the referendum?
(wcar*
(car/bitop "OR" "FATIGUE|HOSTILITY" "FATIGUE" “HOSTILITY")
(car/expire "FATIGUE|HOSTILITY" 10)
(car/bitop "AND" "SCOTLAND&(FATIGUE|HOSTILITY)"
"SCOTLAND" "FATIGUE|HOSTILITY")
(car/expire "SCOTLAND&(FATIGUE|HOSTILITY)" 10)
(car/bitcount "SCOTLAND&(FATIGUE|HOSTILITY)"))

57. How many people in Scotland are Tired
and Grumpy after the referendum?
(wcar*
(car/bitop "OR" "FATIGUE|HOSTILITY" "FATIGUE" “HOSTILITY")
(car/expire "FATIGUE|HOSTILITY" 10)
(car/bitop "AND" "SCOTLAND&(FATIGUE|HOSTILITY)"
"SCOTLAND" "FATIGUE|HOSTILITY")
(car/expire "SCOTLAND&(FATIGUE|HOSTILITY)" 10)
(car/bitcount "SCOTLAND&(FATIGUE|HOSTILITY)"))

58. How many people in Scotland are Tired
and Grumpy after the referendum?
(wcar*
(car/bitop "OR" "FATIGUE|HOSTILITY" "FATIGUE" “HOSTILITY")
(car/expire "FATIGUE|HOSTILITY" 10)
(car/bitop "AND" "SCOTLAND&(FATIGUE|HOSTILITY)"
"SCOTLAND" "FATIGUE|HOSTILITY")
(car/expire "SCOTLAND&(FATIGUE|HOSTILITY)" 10)
(car/bitcount "SCOTLAND&(FATIGUE|HOSTILITY)"))

59. How many people in Scotland are Tired
and Grumpy after the referendum?
(wcar*
(car/bitop "OR" "FATIGUE|HOSTILITY" "FATIGUE" “HOSTILITY")
(car/expire "FATIGUE|HOSTILITY" 10)
(car/bitop "AND" "SCOTLAND&(FATIGUE|HOSTILITY)"
"SCOTLAND" "FATIGUE|HOSTILITY")
(car/expire "SCOTLAND&(FATIGUE|HOSTILITY)" 10)
(car/bitcount "SCOTLAND&(FATIGUE|HOSTILITY)"))

60. How many people in Scotland are Tired
and Grumpy after the referendum?

61. How many people in Scotland are Tired
and Grumpy after the referendum?
HOSTILE 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0

62. How many people in Scotland are Tired
and Grumpy after the referendum?
HOSTILE 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0
FATIGUE 0 1 0 1 0 0 0 0 1 0 0 0 1 0 0 0

63. HOSTILE 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0
FATIGUE 0 1 0 1 0 0 0 0 1 0 0 0 1 0 0 0
BOTH
OR
How many people in Scotland are Tired
and Grumpy after the referendum?

64. HOSTILE 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0
FATIGUE 0 1 0 1 0 0 0 0 1 0 0 0 1 0 0 0
BOTH 0 1 0 1 0 0 0 1 1 0 1 0 1 0 1 0
OR
How many people in Scotland are Tired
and Grumpy after the referendum?

65. HOSTILE 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0
FATIGUE 0 1 0 1 0 0 0 0 1 0 0 0 1 0 0 0
BOTH 0 1 0 1 0 0 0 1 1 0 1 0 1 0 1 0
OR
How many people in Scotland are Tired
and Grumpy after the referendum?

66. How many people in Scotland are Tired
and Grumpy after the referendum?
HOSTILE 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0
FATIGUE 0 1 0 1 0 0 0 0 1 0 0 0 1 0 0 0
BOTH 0 1 0 1 0 0 0 1 1 0 1 0 1 0 1 0
SCT 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0

67. HOSTILE 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0
FATIGUE 0 1 0 1 0 0 0 0 1 0 0 0 1 0 0 0
BOTH 0 1 0 1 0 0 0 1 1 0 1 0 1 0 1 0
SCT 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0
BOTH 0 1 0 1 0 0 0 1 1 0 1 0 1 0 1 0
AND
How many people in Scotland are Tired
and Grumpy after the referendum?

68. HOSTILE 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 0
FATIGUE 0 1 0 1 0 0 0 0 1 0 0 0 1 0 0 0
BOTH 0 1 0 1 0 0 0 1 1 0 1 0 1 0 1 0
SCT 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0
BOTH 0 1 0 1 0 0 0 1 1 0 1 0 1 0 1 0
0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0
2
How many people in Scotland are Tired
and Grumpy after the referendum?

69. RING
Journaling
Syncing
Business logic
What
Where
When
Memory
Web API
Incoming Data

70. RING
Journaling
Syncing
Business logic
What
Where
When
Memory
Web API
Incoming Data
adamwynne/twitter-api

71. RING
Journaling
Syncing
Business logic
What
Where
When
Memory
Web API
Incoming Data
clojurewerkz/meltdown
LMAX-Exchange/disruptor

72. RING
Journaling
Syncing
Business logic
What
Where
When
Memory
Web API
Incoming Data
dakrone/clojure-opennlp

73. “I’m loving #XConfMCR! ”

74. Positive Affect: enthusiastic, active, alert
!
Negative Affect: subjective distress
!
Emerged as distinctive, orthogonal dimensions

75. PANAS
2 3
153 2
4
5
1
1
3
3
3
5
345
2
2
1

76. PANAS
2 3
153 2
4
5
1
1
3
3
3
5
345
2
2
1
1
3
5
9
10
12
14
16
17
19
2
4
6
7
8
11
13
15
18
20

77. PANAS
positive affect! negative affect
3 2
4
5
1
1
5
3
3
1
3 2
5 3
4 3
5 2
2 1
1
3
5
9
10
12
14
16
17
19
2
4
6
7
8
11
13
15
18
20
39 19

78. PANAS-x
General Dimension Scales!
Negative Affect (10)
Positive Affect (10)
Basic Positive Emotions Scales!
Joviality (8)
Self-assurance (6)
Attentiveness (4)
Basic Negative Emotions Scales!
Fear (6)
Hostility (6)
Guilt (6)
Sadness (5)
Other Affective States!
Shyness (4)
Fatigue (4)
Serenity (3)
Surprise (3)

79. PANAS-t
Accounts for bias on social media!
Outlines sanitisation
Validate against 10 real events

80. dakrone/clojure-opennlp
Sanitisation
• Exclude media/advertising/spam
• Account for text speak
• Account for smileys & emoj’s
• Word stemming (or lemmatisation)
• Part of Speech Tagging (POS)
LMAO
GetRichQuick.com

81. SHYNESS
FATIGUE
SERENITY
SURPRISE
FEAR
HOSTILITY
GUILT
SADNESS
JOVIALITY
SELF ASSURANCE
ATTENTIVENESS
We have sentiment keys!

82. RING
Journaling
Syncing
Business logic
What
Where
When
Memory
Web API
Incoming Data
mbostock/d3

83. [-75.14310264, 40.05701649]

84. Reverse Geocoding Issues
• Don’t want external services
• Don’t want heavy IO
• Don’t want round trips to the database
• Accuracy not too much of a concern

87. Equirectangular Projection

88. Courtesy of Eric Gaba (Wikimedia Commons User: Sting)

89. Tissot's indicatrix of deformation
x = λcosφ1
y = φ
Courtesy of Eric Gaba (Wikimedia Commons User: Sting)

91. x (px)
y (px)

92. x (px)
y (px)
[lon, lat] = [55.86140, -4.26477]
[x, y] = [329, 601]
[x, y] = = Scotland!

93. x (px)
y (px)
[lon, lat] = [55.86140, -4.26477]
[x, y] = [329, 601]
[x, y] = = Scotland!

94. x (px)
y (px)
[lon, lat] = [55.86140, -4.26477]
[x, y] = [329, 601]
[x, y] = = Scotland!

95. x (px)
y (px)
[lon, lat] = [55.86140, -4.26477]
[x, y] = [329, 601]
[x, y] = = Scotland!

96. x (px)
y (px)
[lon, lat] = [55.86140, -4.26477]
[x, y] = [329, 601]
[x, y] = = Scotland!

97. x (px)
y (px)
[lon, lat] = [55.86140, -4.26477]
[x, y] = [329, 601]
[x, y] = = Scotland!

98. Ser ver si de!
(JavaFX )
Mike Bostock & Nick Rabinowitz

99. (:require
[hiccup.page :as hiccup])
(:import
(javafx.scene.web WebView))
!
(def scripts ["uk-data.js" "topojson.v1.min.js" "d3.v3.min.js" "geocoder.js"])
!
(defn create-browser [html]
(-> WebView.
.getEngine
(.loadContent html)))
!
(defn build-page [script-paths]
(hiccup/html5
[:head
(for [path script-paths]
Java F X
[:script (slurp (io/resource path))])]))
!
(defn coordinates->name [longitude latitude]
(let [browser (-> scripts build-page create-browser)]
(.executeScript browser
(str "geo.decode([" longitude "," latitude "])"))))

100. (:require
[hiccup.page :as hiccup])
(:import
(javafx.scene.web WebView))
!
(def scripts ["uk-data.js" "topojson.v1.min.js" "d3.v3.min.js" "geocoder.js"])
!
(defn create-browser [html]
(-> WebView.
.getEngine
(.loadContent html)))
!
(defn build-page [script-paths]
(hiccup/html5
[:head
(for [path script-paths]
Java F X
[:script (slurp (io/resource path))])]))
!
(defn coordinates->name [longitude latitude]
(let [browser (-> scripts build-page create-browser)]
(.executeScript browser
(str "geo.decode([" longitude "," latitude "])"))))

101. (:require
[hiccup.page :as hiccup])
(:import
(javafx.scene.web WebView))
!
(def scripts ["uk-data.js" "topojson.v1.min.js" "d3.v3.min.js" "geocoder.js"])
!
(defn create-browser [html]
(-> WebView.
.getEngine
(.loadContent html)))
!
(defn build-page [script-paths]
(hiccup/html5
[:head
(for [path script-paths]
Java F X
[:script (slurp (io/resource path))])]))
!
(defn coordinates->name [longitude latitude]
(let [browser (-> scripts build-page create-browser)]
(.executeScript browser
(str "geo.decode([" longitude "," latitude "])"))))

102. (:require
[hiccup.page :as hiccup])
(:import
(javafx.scene.web WebView))
!
(def scripts ["uk-data.js" "topojson.v1.min.js" "d3.v3.min.js" "geocoder.js"])
!
(defn create-browser [html]
(-> WebView.
.getEngine
(.loadContent html)))
!
(defn build-page [script-paths]
(hiccup/html5
[:head
(for [path script-paths]
Java F X
[:script (slurp (io/resource path))])]))
!
(defn coordinates->name [longitude latitude]
(let [browser (-> scripts build-page create-browser)]
(.executeScript browser
(str "geo.decode([" longitude "," latitude "])"))))

103. (:require
[hiccup.page :as hiccup])
(:import
(javafx.scene.web WebView))
!
(def scripts ["uk-data.js" "topojson.v1.min.js" "d3.v3.min.js" "geocoder.js"])
!
(defn create-browser [html]
(-> WebView.
.getEngine
(.loadContent html)))
!
(defn build-page [script-paths]
(hiccup/html5
[:head
(for [path script-paths]
Java F X
[:script (slurp (io/resource path))])]))
!
(defn coordinates->name [longitude latitude]
(let [browser (-> scripts build-page create-browser)]
(.executeScript browser
Stuart Sier ras
Co mpo nent lib!!
(str "geo.decode([" longitude "," latitude "])"))))

104. RING
Journaling
Syncing
Business logic
What
Where
When
Memory
Web API
Incoming Data
KirinDave/clj-time

105. “Thu Jun 26 14:35:57 +0000 2014”
over 86 thousand seconds in a day
over 31 million seconds in a year

108. mm:260620141435

109. HH:260620141400

110. dd:260620140000
Monday Tuesday Wednesday Thursday Friday Saturday Sunday
1
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 27 28 29
30 31

111. MM:010620140000
JAN FEB MAR APR
MAY JUL AUG
SEP OCT NOV DEC

112. 2013
2015
yy:010120140000

113. (:require
[clj-time.format :as time])
!
!
(def input-formatter "EE MMM d HH:mm:ss Z yyyy")
(def formats { "mm:" "ddMMyyHHmm" "HH:" "ddMMyyHH00" ... })
!
!
(defn create-key [date [prefix formatter]]
(let [out-date (time/unparse formatter date)]
(str prefix out-date)))
!
!
(defn date->keys [created_at]
(let [date (time/parse input-formatter created_at)]
(map #(create-key date %) formats)))

114. (:require
[clj-time.format :as time])
!
!
(def input-formatter "EE MMM d HH:mm:ss Z yyyy")
(def formats { "mm:" "ddMMyyHHmm" "HH:" "ddMMyyHH00" ... })
!
!
(defn create-key [date [prefix formatter]]
(let [out-date (time/unparse formatter date)]
(str prefix out-date)))
!
!
(defn date->keys [created_at]
(let [date (time/parse input-formatter created_at)]
(map #(create-key date %) formats)))

115. (:require
[clj-time.format :as time])
!
!
(def input-formatter "EE MMM d HH:mm:ss Z yyyy")
(def formats { "mm:" "ddMMyyHHmm" "HH:" "ddMMyyHH00" ... })
!
!
(defn create-key [date [prefix formatter]]
(let [out-date (time/unparse formatter date)]
(str prefix out-date)))
!
!
(defn date->keys [created_at]
(let [date (time/parse input-formatter created_at)]
(map #(create-key date %) formats)))

116. RING
Journaling
Syncing
Business logic
What
Where
When
Memory
Web API
Incoming Data
ptaoussanis/carmine

117. We have 11 keys in total!
SENTIMENT
!
LOCATION
!
mm:date-time
HH:date-time
dd:date-time
MM:date-time
yy:date-time
What
Where
When

118. (:require
[taoensso.carmine :as redis])
!
!
(defn create-tweet-id []
(redis/incr "global:tweet.id"))
!
!
(defn set-bits [data-keys tweet-id]
(doseq [data-key data-keys]
(redis/setbit data-key tweet-id 1)))
!
!
(defn data->analytics [data-keys]
(let [tweet-id (create-tweet-id)]
(set-bits data-keys tweet-id)))

119. (:require
[taoensso.carmine :as redis])
!
!
(defn create-tweet-id []
(redis/incr "global:tweet.id"))
!
!
(defn set-bits [data-keys tweet-id]
(doseq [data-key data-keys]
(redis/setbit data-key tweet-id 1)))
!
!
(defn data->analytics [data-keys]
(let [tweet-id (create-tweet-id)]
(set-bits data-keys tweet-id)))

120. (:require
[taoensso.carmine :as redis])
!
!
(defn create-tweet-id []
(redis/incr "global:tweet.id"))
!
!
(defn set-bits [data-keys tweet-id]
(doseq [data-key data-keys]
(redis/setbit data-key tweet-id 1)))
!
!
(defn data->analytics [data-keys]
(let [tweet-id (create-tweet-id)]
(set-bits data-keys tweet-id)))

121. {
'query-type': 'count',
'params': {
'sentiments': [
'FATIGUE',
'HOSTILITY'
],
'locations': [
'SCT'
]
}
}
weavejester/compojure
JSON quer y

122. {
'query-type': 'count',
'params': {
'sentiments': [
'FATIGUE',
'HOSTILITY'
],
'locations': [
'SCT'
]
}
}
weavejester/compojure
JSON quer y
/ap i?_quer y={jso n}

123. (defmulti execute :query-type)
!
!
(defmethod execute “count" []
… logic here)
!
!
(defmethod execute “p-affect“ []
… logic here)
!
!
(defmethod execute “n-affect“ []
… logic here)
{
'query-type': 'count',
'params': {
'sentiments': [
'FATIGUE',
'HOSTILITY'
],
'locations': [
'SCT'
]
}
}
weavejester/compojure
JSON quer y
/ap i?_quer y={jso n}

124. (defmulti execute :query-type)
!
!
(defmethod execute “count" []
… logic here)
!
!
(defmethod execute “p-affect“ []
… logic here)
!
!
(defmethod execute “n-affect“ []
… logic here)
{
'query-type': 'count',
'params': {
'sentiments': [
'FATIGUE',
'HOSTILITY'
],
'locations': [
'SCT'
]
}
}
weavejester/compojure
JSON quer y
/ap i?_quer y={jso n}

125. weavejester/compojure
(defmulti execute :query-type)
!
!
(defmethod execute “count" []
… logic here)
!
!
(defmethod execute “p-affect“ []
… logic here)
!
!
(defmethod execute “n-affect“ []
… logic here)
JSON quer y
{
'query-type': 'count',
'params': {
'sentiments': [
'FATIGUE',
'HOSTILITY'
],
'locations': [
'SCT'
]
}
}
/ap i?_quer y={jso n}

126. weavejester/compojure
(defmulti execute :query-type)
!
!
(defmethod execute “count" []
… logic here)
!
!
(defmethod execute “p-affect“ []
… logic here)
!
!
(defmethod execute “n-affect“ []
… logic here)
JSON quer y
{
'query-type': ‘p-affect',
'params': {
'sentiments': [
'FATIGUE',
'HOSTILITY'
],
'locations': [
'SCT'
]
}
}
/ap i?_quer y={jso n}

127. weavejester/compojure
(defmulti execute :query-type)
!
!
(defmethod execute “count" []
… logic here)
!
!
(defmethod execute “p-affect“ []
… logic here)
!
!
(defmethod execute “n-affect“ []
… logic here)
JSON quer y
{
'query-type': ‘p-affect',
'params': {
'sentiments': [
'FATIGUE',
'HOSTILITY'
],
'locations': [
'SCT'
]
}
}
/ap i?_quer y={jso n}