Slides from presentation at Infinum Android Talks #14

1. Complex Data Models
Real-time maintenance and change integration

2. Hello.
Jovica Popović
Senior Developer
https://www.google.com/+JovicaPopovic
minus5 d.o.o.
http://www.minus5.hr/

3. Case study:
Sporti App
Android application for SuperSport online
betting platform
App available for download at (needs registration):
https://www.supersport.hr/android/SuperSport_Android.apk

4. Case study:
Sporti App
~25k MAU
~800k monthly sessions
~200k monthly live video streams played
up to 2000 concurrent users
3.5+ million betting slips entered
8+ million GCM notifications delivered

5. Sports Betting Data
Model characteristics:
~200 Leagues
~6000 Matches
~20000 Bet offers
~2MB values only JSON
~8MB POJO size
- updates every couple of minutes
["1.0",{"9093620":[172408,8946464,"41","Austria Beĕ(sš)-SCR
Altach","sri 18:00","2015-11-
04",[[503789334,1.6,"1"],[503789335,4.0,"X"],[503789336,8.0,"2"],[50
3789337,1.15,"1X"],[503789338,2.7,"X2"],[503789339,1.35,"12"],[503
789340,2.5,"f+2"]],null,null,null,null,null,6,null,null,"01AABB",null,null,n
ull,null,true],"9093475":[172408,9011786,"42","Cibona(sš)-Antwerp
Giants","sri 19:00","2015-11-
04",[[503789052,1.6,"1"],[503789053,17.0,"X"],[503789054,3.2,"2"],[5
03789055,1.45,"1X"],[503789056,2.7,"X2"]],null,null,"(košarka)",null,
null,6,null,null,"01AABB",null,null,null,null,true],"9093471":[172408,86
79741,"43","Mac.Tel Aviv(sš)-FC Porto","sri 20:45","2015-11-
04",[[503788992,8.0,"1"],[503788993,4.0,"X"],[503788994,1.6,"2"],[50
3788995,2.7,"1X"],[503788996,1.15,"X2"],[503788997,1.35,"12"],[503
788998,2.5,"f+2"]],null,null,null,null,null,...

6. Sports Betting Data
Challenges:
import speed
memory footprint
memory churn (on Dalvik VM)
Solution approaches:
JsonObject
Jackson ObjectMapper
Jackson JsonParser
Separate import process

7. Final JSON parsing implementation:
Parser gets passed onto
contained model classes in the
order objects appear in the
source JSON. Import will break
if the expected order is not
respected, so data format
needs to be strictly enforced -
we’re sacrificing adaptability for
speed.

8. Update handling
Normal operation:
Model updates (diffs) pushed over websocket
● contain ‘from’ and ‘to’ version
● only changes and deletions
● regular AsyncTask merge
Lost sync (connection loss, went to background):
Re-sync using manifest file containing update history
● http fetch of missed diff files
● diff chaining
● full import as a last resort

9. Live Betting Data
Model characteristics:
50-100KB JSON basic index
5-10KB data per match
<100 Matches
<1000 Bet offers
- frequent updates (up to several per second)
Challenges:
fast update rate
multiple model update sources
basic index data (full/diff)
match data (full/diff)
user interaction
business rules logic module

10. Live betting data import
Websocket message streams:
● Index full data
● Index diff data
● Match full/diff stream
● string->JsonObject->POJO
All three feed into RxJava observable:

11. Live betting data presentation
PonudaServiceMsgReceiver generates new model data and notifies business logic module and they
in turn publish into three RxJava Action streams:

12. Assorted bits ’n’ pieces
Miscellania:
● model and websocket kept in separate started Services
● using ObjectPool for biggest memory churners
● keep websocket alive for a couple of minutes after onPause()
● staggered auto re-sync after reconnect in onResume()
● delayed (1 hour) ‘suicide’ after onPause() on Dalvik to get clean heap
● background data load started immediately, before login
● implemented onTrimMemory() to stave off app getting the axe
Room for improvement:
● move sports data onto index/full event model
● break out model Service into a separate process?
● Flatbuffers instead of JSON?

13. That’s all folks. Any Qs?

14. P.S. libs used
Jackson JSON manipulation: com.fasterxml.jackson.core:jackson-core
AndroidAsync HTTP & websocket: com.koushikdutta.async:androidasync
ObjectPool pooling: nf.fr.eraasoft:objectpool
Reactive: io.reactivex:rxandroid
roll-our-own message bus