73. Fallacies of Distributed Computing
1.The network is reliable.
2. Latency is zero.
3. Bandwidth is infinite.
4.The network is secure.
5.Topology doesn't change.
6.There is one administrator.
7.Transport cost is zero.
8.The network is homogeneous.
74. A grounded, peer to peer
replicated, disconnected,
versioned, property
database with self-healing
conflict resolution
Prophet
78. grounded
Runs at the edge
Doesn’t need to run in the cloud
Syncs with services you already use
(Adaptors talk to “Foreign Replicas”)
79. Update any replica
Pull from any replica
Push to any replica
Publish a replica
Changes will propagate
peer-to-peer replicated
80. Real-time replication is hard to scale
It only “works” with constant connectivity
I don’t have constant connectivity
Neither do you
Prophet sync can happen whenever
disconnected
81. Every update is recorded as a change set
Change sets don’t lose any data
(so you can use them to go backwards)
All history is introspectable
Replication just replays changesets
versioned
82. Atomic operations
CREATE, READ, UPDATE, DELETE, SEARCH
Record types can have optional validation
and canonicalization
Records of the same type do not need to
have the same properties
Add and remove properties at will
property database
83. Remembers all conflict resolutions
Syncs all resolutions with your peers
Detects identical conflicts
Uses your peers’ resolutions to “vote” for
the winner of a conflict
self-healing
conflict resolution
85. RESTy API
GET /records.json
GET /records/Cars.json
GET /records/Cars/716499-5F9-4AC4-827.json
GET /records/Cars/716499-5F9-4AC4-827/wheels.json
POST /records/Cars.json
POST /records/Cars/716499-5F9-4AC4-827.json
POST /records/Cars/716499-5F9-4AC4-827/wheels.json
86. RESTy API
Yes, we should be using PUT and DELETE
(or just provide an OpenRESTY adaptor)
Yes, you can have a commit bit and
help us fix it :)
87. Native API
(Yes, the core is Perl.)
my $cli = Prophet::CLI->new();
my $cxn = $cli->app_handle->handle;
my $record = Prophet::Record->new( handle => $cxn, type => 'Person' );
my $uuid = $record->create( props => { name => 'Jesse', age => 31 } );
$record->set_prop( name => 'age', value => 32 );
my $people = Prophet::Collection->new( handle => $cxn, type => 'Person' );
$people->matching( sub { shift->prop('species') ne 'cat' } );
91. ./bin/sd ticket create --
summary="Can't sync sd with Google Code"
status=new
Created ticket 5 (93BF979E-08C1-11DD-94C3-D4B1FCEE7EC4)
Create
92. ./bin/sd ticket search --regex publish
29 } new the online help doesn't describe publish
34 } new publish a static html view of records
35 } new publish should create a static rss file
List and Search
100. (Using only the public REST API)
It took an afternoon
Mirror an RT instance into SD
Share it with your peers using prophet
Sync changes back from your peers to RT
Supports Comments and Attachments
Wrote an RT Replica for SD
114. The Changeset Store
Stores every change to a set of records
Guaranteed to have all old changesets
Replaying all changesets will create an exact
clone of the replica
117. HTTP
Designed to let you “publish” databases
Flat-files, Currently read-only.
Same format as the filesystem replica type.
118. Backends are pluggable!
The filesystem is cheap and easy
The filesystem is portable
Help us write new backends:
CouchDB, Postgres, SQLite, MySQL, S3,
AppEngine, $YOUR_FAVORITE_DB
119. Prophet is designed to sync with “other”
databases and systems
They don’t need to support all of Prophet’s
features - Prophet knows how to interpret
mumbo-jumbo from the Cloud
Foreign Replicas will usually be app specific
All current examples are for SD
Foreign Replicas
125. Figures out the best resolution
“Nullifies” the conflict so the changeset can
be cleanly integrated
Integrates the conflicting changeset
Records the resolution as a new changeset
Records the resolution decision in the
resolution database
Resolving Conflicts
126. Prophet has clever ways to figure out the best
resolution.
If there are previous resolutions for the same
conflict and a majority agree, use that
If the merger has specified a “prefer this side”
choice, use that
Prompt the user to make a decision, giving
them info about previous decisions for this
conflict
“The Best Resolution”
128. Scaling to giant clusters is boring
(Can I play the “They’re not Green” card here?)
Scales to many weakly connected peers
You are not Google
Does anyone here work for Google?
Current target is databases of O(50k) records
How does it scale?
129. We have a political agenda.
Cloud computing is not Open.
APIs for “export” are not good enough.
You should always have full control.
You probably don’t need to store 10 billion
records in one database.
Why not, then?
130. Do you have 10 billion
bugs, customer contacts
or sales orders?
134. Project Status
Simple, well-defined Perl API
RESTy web API (with microserver)
Fast, lightweight backend
Small, active dev community
Great test coverage
...less than great documentation coverage
135. Better ergonomics
Improved search and indexing
(Including full-text indexing)
Client libraries for other languages
Proper security model
More apps
Our Plans
136. Prophet
8225 lines of code and doc
2120 lines of tests
sd
2751 lines of code and doc
1121 lines of tests
Codebase
137. Prophet is very young
Prophet designed in April
Prophet core implemented in April
SD designed in April
SD built in June and July
138. We need your help!
Kick-ass functional and text indexing
Backend data store improvements
Slick GUIs for syncing
More Foreign Replicas for SD
Documentation improvements
A clever logo
New applications