11. When relations are wrong
Sparse data
Irregular data
Poorly understood interrelationships
No definable indexes
Big data
No vertically scalable hardware
12. Papers Read Around the World
Google's BigTable:
http://research.google.com/archive/
bigtable.html
Amazon's Dynamo:
http://www.allthingsdistributed.com/
2007/10/amazons_dynamo.html
13. Lessons from
Functional Programming
MapReduce:
http://research.google.com/archive/
mapreduce.html
14. MapReduce
map(String key, String value):
// key: document name
// value: document contents
for each word w in value:
EmitIntermediate(w, "1");
reduce(String key, Iterator values):
// key: a word
// values: a list of counts
int result = 0;
for each v in values:
result += ParseInt(v);
Emit(AsString(result)); [1]