“Use the right tool for the right job” is one of the first thing they teach you when you start out in these waters. I would make “Get to really know your tools” a second.
In this talk we’re going to work on the architecture of an app that showcases some common features/scenarios we all probably already have in the apps we’re working on: counters, leaderboards, queuing, timelines, caching. But this time we’ll implement them with Redis, making the apps much faster, your hardware (and you) much cooler, your boss (and the clients) much happier and hopefully your salary a bit higher.
8. AVAILABLE CLIENTS IN:
ActionScript bash C C# C++ Clojure
Common lisp Crystal D Dart Elixir emacs lisp
Erlang Fancy gawk GNU Prolog Go Haskell
Haxe Io Java Javascript Julia Lua
Matlab mruby Nim Node.js Objective-C OCaml
Pascal Perl PHP Pure Data Python R
Racket Rebol Ruby Rust Scala Scheme
Smalltalk Swift Tcl VB VCL
9. AVAILABLE DATA STRUCTURES
▸ Strings (Binary safe, can be anything from"hello world"to a jpeg file)
▸ Lists (Collections of string elements sorted according to the order of insertion)
▸ Sets (Collections of unique, unsorted string elements)
▸ Sorted sets (It's like Sets with a score)
▸ Hashes (Maps of fields associated to values. Think non-nested json objects)
▸ Bitmaps (Manipulate Strings on a bit level)
▸ HyperLogLogs (Probabilistic data structure used to estimate the cardinality of a set)
11. TWITTER ANALYSIS TOOL
▸ Track a selected group of hashtags (#gameofthrones, #got, #gotseason7)
▸ Count mentions of certain keywords ('winter is coming', 'tyrion', 'jon snow', 'stark', 'targaryen', 'cersei', 'asha greyjoy', 'Khaleesi', 'sansa', 'arya')
METRICS:
▸ A feed of all tweets containing one of the hashtags
▸ Total number of tweets with one or more of the selected hashtags
▸ A leaderboard of keyword frequency
▸ A feed of tweets per keyword
15. [2] SET TRACKED DATA
Use sets to store all the hashtags we'll be looking at and all the keywords
as well
$client->sadd('hashtags', 'gameofthrones','got', 'gotseason7');
// hashtags | 'gameofthrones'
// | 'got'
// | 'gotseason7'
$client->sadd('keywords', 'winter is coming', 'winterfell', 'jon snow',
'stark', 'targaryen', 'cersei', 'asha greyjoy', 'dorne', 'Khaleesi'
'hodor', 'sansa', 'arya', 'white walkers', 'the night king');
17. [3] GET THE DATA
Use Twitter Stream API to receive notifications for tweets containing
any of the hashtags we're following
$hashtags = $client->smembers('hashtags');
// array (size=3)
// 0 => string 'got' (length=3)
// 1 => string 'gameofthrones' (length=13)
18. Save every new tweet from the stream as a separate String.
$keyname = 'tweet_id:' . $tweet_id;
$tweet_contents = "Winter is coming Khaleesi! #gameofthrones";
$client->set($keyname, $tweet_contents)
// 'tweet_id:45645656' | 'Winter is coming Khaleesi! #gameofthrones'
And then push to a queue to be processed asynchronously
// Use the list data structure as a queue
$client->lpush('message_queue', $keyname);
// 'message_queue' | 'tweet_id:45645656'
// | 'tweet_id:44645234'
// | 'tweet_id:43645232'
20. [4] WORKER TO PROCESS THE QUEUED JOBS
A separate worker will be grabbing jobs off the top of the queue and processing them:
$message_queue = $client->rpop('message_queue');
// 'message_queue' | 'tweet_id:45645656'
// | 'tweet_id:44645234'
// | 'tweet_id:43645232'
Reliable queue: RPOPLPUSH, BRPOPLPUSH
Blocking queue: BLPOP, BRPOP