This talk discusses big data and what it can do for Sri Lanka, learning from efforts like Code for America.

1. What Open Data and
Open Source can do
for Sri Lanka?
Srinath Perera, Ph.D
VP Research, WSO2
Member, Apache Foundation
@srinath_perera

2. A Day in Your Life

3. Success Stories
• Money Ball ( Baseball drafting)
• Nate Silver predicted outcomes in 49 of
the 50 states in the 2008 U.S. Presidential
election
• Cancer detection from Biopsy cells ( Big
Data find 12 patterns while we only knew
9), http://go.ted.com/CseS
• Bristol-Myers Squibb reduced the time it
takes to run clinical trial simulations by
98%
• Xerox used big data to reduce the attrition
rate in its call centers by 20%.
• Kroger Loyalty programs ( growth in 45
consecutive quarters)

4. If you collect data about your business, and feed it to a Big Data
system, you will find useful insights that will provide competitive
advantage
– (e.g. Analysis of data sets can find new correlations to "spot business
trends, prevent diseases, combat crime and so on”. [Wikipedia])

5. Putting Analytics to Work
 What happened? And
Why? ( Hindsight)
 What is Happening
right now? (
oversight)
 What will happen?
(Foresight)

6. Open Source Market Share
• Apache (60%)
• Linux (Servers 16%)
• Firefox (25%)
• Tomcat and most of
middleware
• Android (43%)
• Even Microsoft looking
favorably at Opensource
projects
• There are lot of open
source projects bundled
inside the proprietary
products
Copyright kafka4prez and licensed for reuse under CC License ,
http://www.flickr.com/photos/kafka4prez/198465913

7. What is Open Source?
• Most commercial software does
not distribute the source code, and
developed and managed in a
closed world.
• Idea of open source is to have the
code in the open, and to improve
it though volunteer contributions
using “open development”
• Idea is that the project becomes a
eco-system
– More ideas
– More developers
– More Testers
– More Bug fixers
“There is no delight in
owning anything
unshared.”
Seneca (Roman philosopher,
mid-1st century AD)

8. How does a Open Source Work?
• Open code repository (SVN or Git
etc.)
• Two parts of the community
– Developer Community
– User Community
• Communication through Mailing
lists / IRC Channel
– Develop mailing list
– User mailing list
• Bug tracking database to track errors
(Jira, Bugzilla)
• People submit improvements as
patches through Jira etc.
 Committers have write access to repository
 Committers review and apply patches, and when you
submit lot of them, they will make you a committer.

9. History of Opensource
• 1970s – UNIX, Emacs
• 1984-85 - GNU project and
Free Software Foundation
• 1990 - GNU project almost
complete .. well not OS
• 1991 - Linus Torvalds announce
Linux, Phython
• 1993 - Net BSD and Free BSD
• 1994-95 - Linux 1.0 released
• 1995 - Apache, KDE, PHP
• 1997 - Genome
• 1999 Linux 2.2, OpenOffice
• 2003 - Firefox, Android
http://www.geograph.org.uk/photo/916456
http://www.fotopedia.com/items/flickr-3320704544

10. Why People Contribute?
• As a way to improve your profile
(looking for a Job)
• To gain experience
• To work with “like minded” People
• To be part of something bigger
• To be a “Geek”
• As a Job – if you a well known
open source developer, chances are
that you will get payed for
contribution
• As a competitive strategy
Copyright U. S. Fish and Wildlife Service and licensed for reuse under CC License ,
http://www.flickr.com/photos/usfwsnortheast/4754624921 and Copyright WxMom and licensed
for reuse under CC License , http://www.flickr.com/photos/wxmom/1359996991.

11. • Sahahna
• Apache Axis2 and
other projects
http://www.geograph.org.uk/photo/1842872
LKA Success Stories

12. Why People use Open Source
Software?
• It is cheaper
• It is better
• Because it is open source (Religiously)
• More visibility into the code, better security,
auditing
• If there is a problem, I can fix it
• More control over releases, roadmap
• Patches become available faster
• Easy to understand how it works
• Can fork the code if needed
• Not own by one person, less risk to depend
on it.
• Do not have to maintain the code

13. Big Data and Opensource
 Most Big data tools are free
 Even the state of the art is
being released as opensource
 Give countries like a unique
opportunity with a level
playing field

14. Open Data
 Make the data public
 Advanced form of the RTI act
 Opensource idea applied to data science
 E.g. programs like “Code for America”

15. Code Red: US healthcare.gov
Rescue
 $300M project, that is failing
and small group of volunteers
go to hackathon mode to fix
it, and fix it.
 See
 http://radar.oreilly.com/2014/03/cod
e-red_-they-have-no-use-for-someone-
who-looks-and-dresses-like-me.html
 http://content.time.com/time/magazi
ne/article/0,9171,2166770-1,00.html

16. Filtering Information with Big
Data Big Data can filter
information (e.g. SPAM)
 Rank Information ( show
most relevant articles)
 Find Anomalies ( detect
Fraud)
 Make recommendations (
product
recommendations)
 Handle reputations (e.g.
Ebay, Amazon)
George Caleb Bingham, 1846

17. Example: Reddit, Hacker
News( Ranking)
 Keep Your
Customers
 Get New Customers
 Improve Operations
 Monetize your data

18. Example: Amazon, Ebay,
Abnb, (reputation)

19. Urban Planning and Policy
Decisions
• Urban Planning
– People distribution
– Mobility
– Waste Management
– Parking
• Policy Decision
– What if we change
minimum wage?
– What are economic impact
of a new law?
By Aqwis - Own work, CC BY-SA 3.0,
https://commons.wikimedia.org/w/index.php?curid=6810430

20. Example: Big Data for
Development
• Done using CDR data
• People density noon vs. midnight
(red => increased, blue =>
decreased)
From: http://lirneasia.net/2014/08/what-does-big-data-say-about-sri-lanka/

21. Traffic
 Lot of us waste time on
traffic
 Know where is traffic (
Google traffic does that)
 Emergency Response
 Know the traffic patterns
 Long term planning

22. Manage Donors and
Charities
 Sri Lanka donates a lot (even the poorest)
 Does the money goes to intended place
 Can we track how money is spent?
https://iwringer.files.wordpress.com/2015/09/
traffic2.jpg?w=656

23. Day to day Maintenance
 Does the news papers are the best way to get day to
day things done?
 Can crowd sourcing help?
 How to stop false tickets?

24. Disease spread
 Earlier Malaria and now dengue
 Know current situation
 Know overall trends ( focus on problematic
areas)
 Emergency Response

25. Summary
• There are lot Opensource, Open
data, and Big Data can do for Sri
Lanka
• Some cases needs money!! And
might be beyond us
• But not for many cases
– e.g. Sahana
– Hackathon to build an app to decide
what topics to take up in the
parliament
• What we really need is
collaborations between domain
experts and computer scientists