Rethinking Syncing at AltConf 2019

RETHINKING
SYNCING
AltConf 2019

Syncing Variables
• Amount of Data
• When does user want to see or use data
• How much / how often does data change
• Offline access needed

REST API
https://sync.server.com/green/GET
GET
PUT
POST
DELETE
https://sync.server.com/green/123

To-Device Syncing Approaches
• The Everything Approach
• The Net Change Approach
• The Kinda Sorta Not Really Net Change Approach
• The As Needed Approach
• and more…

The Net Change Approach
https://sync.server.com/allmystuff?since=20180211

• Updates data from known point in time
• More efficient implementation, more complex logic required
• Uses less network bandwidth
• Requires handling of adds, changes, deletes on server and mobile
• Requires one source of truth for “since” parameter
https://sync.server.com/allmystuff?since=20180211

• Process:
• Make API calls - receive lists of “red,” “green,” and “blue” objects with actions identified
• Iterate each list - perform add / change / delete as needed to local storage
• Update UI

• Prepare API Call:
static func urlRequest(for url:URL, parameters:[String:String]?) -> URLRequest {
var requestUrl = url
if let passedParameters = parameters {
var queryItems:[URLQueryItem] = []
for (parameterName, parameter) in passedParameters {
queryItems.append(URLQueryItem(name: parameterName, value: parameter))
}
var requestComponents = URLComponents(url: requestUrl, resolvingAgainstBaseURL: false)
if let currentQueryItems = requestComponents?.queryItems {
queryItems.append(contentsOf: currentQueryItems)
}
requestComponents?.queryItems = queryItems
if let queryRequestUrl = requestComponents?.url {
requestUrl = queryRequestUrl
}
}
let request = URLRequest(url: requestUrl)
return request
}

• Make API Call:
static func fetchObjects<T:RemoteObject>(of type:T.Type,
completionHandler: @escaping (_ results:[T]) -> Void,
errorHandler: @escaping (_ error:Error) -> Void) -> Void {
let fetchURL = type.urlForList(from: AppEnvironment.APIBaseURL)
let fetchTask = APIManager.authSession?.dataTask(with: type.urlRequest(for: fetchURL, parameters: nil),
completionHandler: { (data, response, error) in
if let actualError = error {
DispatchQueue.main.async {
errorHandler(actualError)
}
return
}
if let httpResponse = response as? HTTPURLResponse, httpResponse.statusCode != 200 {
errorHandler(NetworkingError.unexpectedHTTPStatusCode)
}
return
}
…
})
fetchTask?.resume()
}

• Ingest Data:
do {
guard let payloadData = data,
let payload = try JSONSerialization.jsonObject(with: payloadData, options: []) as? [String:Any],
let resultsPayload = payload["results"] as? [[String:Any]] else {
errorHandler(NetworkingError.invalidJSONPayload)
}
return
}
let results = T.ingest(json: resultsPayload)
completionHandler(results)
}
} catch {
}
return
}

do {
}
return
}
}
} catch {
}
return
}
}
return
}
}
return
}
…
})
fetchTask?.resume()
}
static func urlRequest(for url:URL, parameters:[String:String]?) -> URLRequest {
var requestUrl = url
if let passedParameters = parameters {
var queryItems:[URLQueryItem] = []
for (parameterName, parameter) in passedParameters {
queryItems.append(URLQueryItem(name: parameterName, value: parameter))
}
var requestComponents = URLComponents(url: requestUrl, resolvingAgainstBaseURL: false)
if let currentQueryItems = requestComponents?.queryItems {
queryItems.append(contentsOf: currentQueryItems)
}
requestComponents?.queryItems = queryItems
if let queryRequestUrl = requestComponents?.url {
requestUrl = queryRequestUrl
}
}
let request = URLRequest(url: requestUrl)
return request
}
do {
}
return
}
}
} catch {
}
return
}
}
return
}
}
return
}
…
})
fetchTask?.resume()
}
do {
}
return
}
}
} catch {
}
return
}
let fetchTask = APIManager.authSession?.dataTask(with: type.urlRequest(for: fetchURL, param
}
return
}
}
return
}
…
})
fetchTask?.resume()
}

From-Device Syncing Approaches
• The Online Only Approach
• The Persist-The-Network-Call Approach
• The Offline First Queue Approach
• and more…

Complications to REST API Syncing
• Offline mode
• Queueing uploads
• Dependencies
• Must fetch one entity before fetching another
• Must create one entity before creating another
• Model changes
• Sync API changes with releases
• Support older versions

We didn’t
need to write
syncing
code…
any more?!

Eventually Consistent DB
• Server Database (can be NoSQL)
• Sync Server
• Mobile DB

Eventually Consistent DBs
• Examples:
• Couchbase
• Realm
• Google Firebase
• AWS AppSync
• IBM Cloudant
• others?

The idea:
Couchbase Server
Sync Server
Legacy DB
Services
Couchbase Lite

The idea:
Couchbase Server
Couchbase Lite
Sync Server
Legacy DB
Services

What just happened:
Couchbase Server
Sync Server
Legacy DB
Services
Couchbase Lite

How do we implement?
• Model Objects
• Data Controller
• Fetching Data for Display
• Handling Refresh on Change
• Adding Data
• Deleting Data

Model Objects
struct BlueBar:Codable {
var userId: String
var blueIntDetail: Int
var type: String
var addedBy: String
}
struct RedBar:Codable {
var userId: String
var redIntDetail: Int
var type: String
var addedBy: String
}
struct GreenBar:Codable {
var userId: String
var greenIntDetail: Int
var type: String
var addedBy: String
}

Data Controller
init() {
do {
database = try Database(name: "rethinking")
} catch {
print("Unable to initialize Couchbase database 'rethinking'")
}
}

Data Controller (cont)
func setUpSyncing(at url:URL) {
guard let database = database else {
print("No Couchbase database, cannot start syncing")
return
}
let endpoint = URLEndpoint(url: url)
let config = ReplicatorConfiguration(database: database, target: endpoint)
config.replicatorType = .pushAndPull
config.continuous = true
config.authenticator = BasicAuthenticator(username: "demo_user", password: "password")
config.channels = ["demo_user"]
self.replicator = Replicator(config: config)
//Set up replication change listeners
//Database.setLogLevel(.verbose, domain: .replicator)
//Database.setLogLevel(.verbose, domain: .network)
self.replicator?.start()
}

Fetching Data for Display
func fetchGreenBars() throws -> [GreenBar] {
let documentType = "greenbars"
var fetchedGreenBars: [GreenBar] = []
guard let database = self.database else {
throw DataControllerModelError.noDatabaseError
}
let greenBarQuery = QueryBuilder.select(SelectResult.all())
.from(DataSource.database(database))
.where(Expression.property("type").equalTo(Expression.string(documentType)))
if greenBarQueryListenerToken == nil {
greenBarQueryListenerToken = greenBarQuery.addChangeListener({ [weak self] (change) in
self?.greenBarRefreshHandler?()
})
}
do {
for result in try greenBarQuery.execute() {
let resultDict = result.toDictionary()
if let dataDict = resultDict["rethinking"] {
let data = try JSONSerialization.data(withJSONObject: dataDict, options: .prettyPrinted)
let decodedObject = try self.jsonDecoder.decode(GreenBar.self, from: data)
fetchedGreenBars.append(decodedObject)
}
}
} catch {
print(error)
}
return fetchedGreenBars
}

Handling Refresh on Change
dataController?.greenBarRefreshHandler = { [weak self] in
do {
self?.greenBars = try self?.dataController?.fetchGreenBars() ?? []
} catch {
print("Encounted error attempting to fetch bar records")
}
self?.reloadSection(for: .green)
}

Adding Data
func add(greenBar: GreenBar) throws {
}
let doc = MutableDocument(id: greenBar.userId)
let data = try jsonEncoder.encode(greenBar)
guard let json = try JSONSerialization.jsonObject(with: data) as? [String:Any] else {
throw DataControllerModelError.cannotSerializeJSONError
}
doc.setData(json)
try database.saveDocument(doc)
print("Added green bar: (greenBar.greenIntDetail)")
}

Deleting Data
func delete(greenBar: GreenBar) throws {
}
guard let documentToDelete = database.document(withID: greenBar.userId) else {
throw DataControllerModelError.notFoundError
}
try database.deleteDocument(documentToDelete)
}

struct BlueBar:Codable {
var userId: String
var blueIntDetail: Int
var type: String
var addedBy: String
}
struct RedBar:Codable {
var userId: String
var redIntDetail: Int
var type: String
var addedBy: String
}
struct GreenBar:Codable {
var userId: String
var greenIntDetail: Int
var type: String
var addedBy: String
}
init() {
do {
database = try Database(name: "rethinking")
} catch {
print("Unable to initialize Couchbase database 'rethinking'")
}
}
func fetchGreenBars() throws -> [GreenBar] {
let documentType = "greenbars"
var fetchedGreenBars: [GreenBar] = []
}
let greenBarQuery = QueryBuilder.select(SelectResult.all())
.from(DataSource.database(database))
.where(Expression.property("type").equalTo(Expression.string(documentType)))
if greenBarQueryListenerToken == nil {
greenBarQueryListenerToken = greenBarQuery.addChangeListener({ [weak self] (change) in
self?.greenBarRefreshHandler?()
})
}
do {
for result in try greenBarQuery.execute() {
let resultDict = result.toDictionary()
if let dataDict = resultDict["rethinking"] {
let data = try JSONSerialization.data(withJSONObject: dataDict, options: .prettyPrinted)
let decodedObject = try self.jsonDecoder.decode(GreenBar.self, from: data)
fetchedGreenBars.append(decodedObject)
}
}
} catch {
print(error)
}
return fetchedGreenBars
}
dataController?.greenBarRefreshHandler = { [weak self] in
do {
self?.greenBars = try self?.dataController?.fetchGreenBars() ?? []
} catch {
print("Encounted error attempting to fetch bar records")
}
self?.reloadSection(for: .green)
}
func add(greenBar: GreenBar) throws {
}
let doc = MutableDocument(id: greenBar.userId)
let data = try jsonEncoder.encode(greenBar)
guard let json = try JSONSerialization.jsonObject(with: data) as? [String:Any] else {
throw DataControllerModelError.cannotSerializeJSONError
}
doc.setData(json)
try database.saveDocument(doc)
print("Added green bar: (greenBar.greenIntDetail)")
}
func delete(greenBar: GreenBar) throws {
}
guard let documentToDelete = database.document(withID: greenBar.userId) else {
throw DataControllerModelError.notFoundError
}
try database.deleteDocument(documentToDelete)
}

Eventually Consistent DB Advantages
• Significant simplification of syncing code
• Set up DB, set up syncing call. ~10-20 lines of code
• No API calls needed*
• Dependencies much easier to handle
• Can use nested objects
• Thread safety - can ingest model objects in background and
*safely* use them in main queue. Try doing that in Core Data!
* You might need to do authentication / login with an API call prior to setting up your sync

Eventually Consistent DB Advantages (con’t)
• Significant simplification of model & ingestion code
• No Core Data model needed
• No mogenerated ManagedObjectSubclasses
• Models can be classes or structs using Codable - very simple
to code
• Simplification of services
• Fewer, less complex endpoints

Eventually Consistent DB Disadvantages
• Lack of familiarity
• Example: client refreshed source DB for staging - crushed our
CB instance
• Architecture requires different thinking for common use cases
• Cannot control priority of syncing - “eventual consistency”
• No NSFetchedResultsController-type code (at least in
Couchbase Mobile)

Eventually Consistent DB Best Use Cases
• Handle slowly changing “master” data
• For example, organizing data or lists of choices
• User created data, especially offline
• * Not necessarily images…
• Not ideal for large amounts of quickly changing data

THANK YOU
martiancraft.com
AltConf 2019

Rethinking Syncing at AltConf 2019

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Rethinking Syncing at AltConf 2019

Semelhante a Rethinking Syncing at AltConf 2019 (20)

Último

Último (9)

Rethinking Syncing at AltConf 2019