1.
The DNA of a
GREAT API
Ciprian Sorlea, CTO
October 2015
Building API’s that developers will love

2.
06.11.2015
2
What’s an API:
Any API:
• allows Software components, systems and subsystems to interact.
• hides the complexity of the underlying architecture and functionality
• focuses on separation of concerns
• allows callers/consumers to access resources (ex. CRUD operations) or to
perform various operations on top of the resources

3.
06.11.2015
3
Qualities of a good API:
A good API must be:
• domain specific, simple and easy to use
• consistent (naming, parameters, etc.)
• stateless
• transparent & clear in terms of error trapping
• accessible (from different platforms, languages, etc.)

4.
06.11.2015
4
From Good To Great:
A great API must be:
• well documented and should provide many working samples
• fast, secure, scalable and reliable
• forward and backwards compatible (versioning)
• “self aware”: usage statistics, logs/traces

5.
06.11.2015
5
From Good To Great:
A great API must
go the extra mile

6.
06.11.2015
6
Who are the consumers?
An API can be consumed by:
- A web application
- A desktop application
- A mobile application
- Another API
- An integration test
- A 3rd party integration
- Internet Of Things
- A home appliance, a wearable device, a smart gadget

7.
06.11.2015
7
Now, let’s remember….
Who is your target
audience?
Developers

8.
06.11.2015
8
A big question & challenge:
How do we handle all
these uses cases, while
also making it easy for
Developers?

9.
06.11.2015
9
Things to consider:
- Transport: how do we exchange data?
- Resource models: what kind of data do we exchange?
- API Architecture patterns: how do we design our API?
- Error Handling: how do we handle errors & failure?
- Testing: how do we make our API bullet proof?
- Security: how do we protect our data (or ourselves)?
- Manageability: how can we manage our API and support it operation-wise?
- Evolution: how do we ensure our API is future-proof?
- Adoption: how do we make sure our API will be adopted?

10.
API Transport
how do we exchange data?

11.
06.11.2015
11
API Transport
REST API:
- Recommended for external integrations and applications, 3rd party apps and
plugins
- Use JSON & XML over HTTP(s)
- Use established Security mechanisms (OAuth2, Basic Authentication, etc.)

12.
06.11.2015
12
API Transport
RPC / Binary Protocol:
- Recommended for internal usage or high throughput requirements, IOT and
devices with limited networking capabilities / bandwidth
- Use Apache Thrift, Google Protobuff or alternatives
- Use simplified authentication and authorization model to minimize overhead
– assume working in safer environment

13.
Note: For now,
we’ll focus on
REST API’s

14.
API Resource
Models
what kind of data do we
exchange?

15.
06.11.2015
15
API Resource Models
Resources are the heart and soul of the API
Resources represent the abstraction of any information we exchange and
operate on within an API
Domain Model ~= Resource Model

16.
06.11.2015
16
API Resource Models
Resource classification:
- Singleton – can be part of a collection or can be stand-alone
- ServerInfo – Server Information can be exposed as a standalone singleton resource
- User – User information can be exposed as a resource, part of a collection
- Collection – a list of so called documents belonging to the same domain
- Users
- Products
- Orders
- Documents

17.
06.11.2015
17
API Resource Models
Minimal Resource Model Attributes:
- URI: Unified Resource Identifier
- represents the unique URL by which a resource can be accessed
- HATEOAS takes this to the extreme level
- Ex: http://api.domain.com/v1/serverInfo, http://api.domain.com/v1/users/1,
http://api.domain.com/v1/projects/
- ID:
- required when the resource is a singleton as part of a collection
- a numeric ID (ex: 1029) or a GUID (ex. c366728d-ca93-4b23-a4b9-ae360600b86a)
- can be the database specific primary key (ex. auto generated) or can use a custom field with
random generated values, to enhance security

18.
06.11.2015
18
API Resource Models
Important consideration:
- Ensure your resource models can be properly represented in any transport
models (ex. JSON, XML) and without significant effort (Ex. Don’t try to use
complicated bit level data in your models)
- Follow the Open Close principle. This will make your resources future-proof
- Ensure the API resource model doesn’t carry with it the DB optimizations and
complexity

19.
API Architecture
Patterns
(archetypes)
how do we design our API?

20.
06.11.2015
20
API Archetypes
The de-facto API Archetypes:
• Document – a singleton resource
• Collection – a collection of resources
• Repository (also known as Store) – a collection of immutable resources
• Controller – an action dispatcher ***
*** Should be avoided. Explanation later.

21.
06.11.2015
21
API Archetypes
Pseudo API Archetypes:
• Queue – asynchronous processing of a long operation – can also be a batch operation
or a transaction, but not limited to those
• Should provide status and progress information
• Ex: File processing, order processing, etc.
• Transaction – multiple resource operations to be processed in a ALL-OR-NONE manner
• The request fails if any of the operation fails.
• Any simple action (see Controller) can be also seen/handled as a transaction
• Batch – a sequence of resource operations to be processed in sequence
• Any failure is reported to the consumer, but doesn’t stop the processing of the batch
• Should provide status and progress information
• Ex. Batch create, batch update, etc.

22.
06.11.2015
22
API Archetypes: Document
• Exposes functionality specific to a singleton resource.
• Some standalone documents (ex. ServerInfo), can be read-only
• Some collection-owned documents are mutable (can be updated, deleted,
etc.) -> parent archetype is a Collection

23.
06.11.2015
23
API Archetypes: Document – Partial Retrieval
Partial retrieval allows consumer to limit the transferred information. This leads
to better performance and improved customer experience.
Limiting the fields:
• Specify the fields to be included, as a query string parameter
• Specify the fields to be ignored, as a query string parameter
• Specify the fields to be expanded (sub-details), as a query string parameter
• Specify a predefined view (predefined or custom definition of data retrieval
model)

24.
06.11.2015
24
API Archetypes: Document – Partial Retrieval
Examples:
http://api.domain.com/v1/users?fields=id,firstName,lastName,email,addresses(city)
http://api.domain.com/v1/users?excludedFields=email
http://api.domain.com/v1/users?view=minimalView
http://api.domain.com/v1/users?expand=addresses,previousEmails

25.
06.11.2015
25
API Archetypes: Document – Partial Update
Allows consumers to update only those fields that need to be changed
This provides faster and better user experience.
Two ways:
- Specifying the fields to be changed with the new value (partial documents)
- Using operation maps

26.
06.11.2015
26
API Archetypes: Document – Partial Update
Using partial documents:
PATCH /v1/users/1029 HTTP/1.1
Content-Type: application/json
X-API-PUBLIC-KEY: D6DB835A-35B1-42B3-8805-8F00F5344BB9
X-API-REQUEST-ID: 5BF75F7E-0A55-40DF-9E79-354B9B162971
{
“emailAddress”: ciprian.sorlea@nordlogic.com,
“companyName”: “NordLogic”
“phoneNumbers” : [“+4074ILOVEAPIS”]
}

27.
06.11.2015
27
API Archetypes: Document – Partial Update
Advantages of using partial documents:
- Request very clean and minimal
- Requires very little extra effort to implement
Disadvantages:
- Hard to add items to lists / arrays, remove values from maps, etc.

28.
06.11.2015
28
API Archetypes: Document – Partial Update
Using operation maps:
PATCH /v1/users/1029 HTTP/1.1
Content-Type: application/json
X-API-PUBLIC-KEY: D6DB835A-35B1-42B3-8805-8F00F5344BB9
X-API-REQUEST-ID: 5BF75F7E-0A55-40DF-9E79-354B9B162971
{
“emailAddress”: { “set”: “Ciprian.Sorlea@nordlogic.com”},
“companyName”: { “set” : “NordLogic”},
“phoneNumbers” : { “add” : [“+4074ILOVEAPIS”] }
}

29.
06.11.2015
29
API Archetypes: Document – Partial Update
Advantages of using operation maps :
- More powerful control over what operations are made
- Doesn’t require pre-fetching of documents prior to update to get the actual
map/list content
Disadvantages:
- Requests tend to be more verbose, even though just slightly
- More effort is needed to implement support for this approach (usually)

30.
06.11.2015
30
API Archetypes: Document – Partial Update
PRO TIP: You can implement Partial Update
with either approach, or you can combine
them and allow patching in either way.

31.
06.11.2015
31
API Archetypes: Document – Partial Retrieval
PRO TIP: Combining Partial Update with
Partial Retrieval, APIs can offer extreme
flexibility for a wide variety of consumers &
devices.

32.
06.11.2015
32
API Archetypes: Document – Partial Retrieval
PRO TIP: You can implement transactions
for updating fields that require additional
details on change:
Ex. Change status + comment

33.
06.11.2015
33
Document Archetype: URI & HTTP Verbs
• Base URI: http://api.domain.com/{version}/{documentID} or
http://api.domain.com/{version}/{collection}/{documentID}
HTTP Verb Description
GET Retrieves the document (complete or partial)
PUT Updates a document with the content provided. Only for Collections.
PATCH Updates a document fields specified in the content, or applying the
specific update operations. Only for Collections.
DELETE Deletes the document from the location
HEAD Verifies if the given document id exists.
OPTIONS Lists the available operations / HTTP Verbs

34.
06.11.2015
34
API Archetypes: Collection
Collections provide access to multiple document resources
Operations are usually CRUD
Collection capabilities:
- Pagination
- Filtering: using simple filtering or a query language)
- Sorting: simple (based on single field) or complex (based on multiple fields)
- Partial field retrieval: identical to document level partial field retrieval

35.
06.11.2015
35
API Archetypes: Collection Query String
Query String parameters are used to configure collection functionality (ex:
sorting, filtering, etc.)
Rules & recommendations:
- Use same QS parameter names across your entire API, and document them
- Ensure you don’t introduce un-necessary constraints by choosing improper names
- As long as you don’t confuse your consumers, you can allow aliases for some parameters
Ex:
- Page & PageSize work for a basic pagination but not for continuous scrolling – improper naming
- Start & limit work for any data pagination scenario
- Offset/From and count can be aliases to start & limit

36.
06.11.2015
36
API Archetypes: Collection Query String
Pagination
Ex:
http://api.domain.com/v1/users?from=100&count=10
http://api.domain.com/v1/users?count=10 (uses a default offset/start)

37.
06.11.2015
37
API Archetypes: Collection Query String
Sorting
- can be simple (single field + direction) or complex (list of fields, with specific
sort orders)
- can be applied on 1st level fields or on details (expanded content)
Ex:
http://api.domain.com/v1/users?sortBy=email,sortOrder=DESC
http://api.domain.com/v1/users?orderBy=name:ASC,email:DESC,contactInfo.phone

38.
06.11.2015
38
API Archetypes: Collection Query String
Filtering:
- Simple filters (all filters are of type EQUALS):
http://api.domain.com/v1/users?email=Ciprian.Sorlea@nordlogic.com
http://api.domain.com/v1/users?companyName=Nordlogic&role=developer
- Advanced filters (specify the operation: EQ, NE, GT, LTE, etc.)
http://api.domain.com/v1/users?companyName=LIKE:*logic&role=NE:qa
- Query (use a pseudo query language):
http://api.domain.com/v1/users?query=companyName like ‘*logic’ and role not eq ‘QA’
http://api.domain.com/v1/users?query=name eq ‘John’ or name eq ‘Bill’

39.
06.11.2015
39
Collection Archetype: URI & HTTP Verbs
• Base URI: http://api.domain.com/{version}/{collection}/
HTTP Verb Description
GET / Retrieves a list of paged, filtered and sorted documents, complete or
partial
GET /{docID} Retrieves the document (complete or partial)
POST / Creates a new document in the collection
HEAD Can be used to check if data exists given the provided filters and paging
DELETE Purges the collection. Recommended for Development only.
OPTIONS Lists the available operations / HTTP Verbs

40.
06.11.2015
40
Collection Archetype: Response
Few tips:
- Always echo the pagination details (ex. offset & count), to provide context
- Give more details about the rest of the data (is there more?). Ex: maxRows
- Don’t include empty/null fields and ensure (through documentation) your
consumers handle that. (This applies to single documents as well)

41.
06.11.2015
41
API Archetypes: Repository
Repositories provide access to multiple immutable document resources
Operations are usually CRD (Create, Retrieve, Delete)
Updates to documents within Repositories MIGHT happen behind the API level
Nice to have capabilities:
- Capped repositories: limit data to most recent X records
- Auto Expiration: automatically delete data older than X days/hours/minutes
- Tailing options: retrieve latest X records

42.
06.11.2015
42
Repository Archetype: URI & HTTP Verbs
• Base URI: http://api.domain.com/{version}/{repository}/
HTTP Verb Description
GET / Retrieves a list of documents with limited capabilities (ex. Tailing over
the records in an audit/trace log). Nov very common.
GET /{docID} Retrieves the document (complete or partial).
POST Creates a new document in the repository
HEAD When GET is also implemented, this can be used to check if data exists.
DELETE Purges the repository. Recommended for Development only.
OPTIONS Lists the available operations / HTTP Verbs

43.
06.11.2015
43
API Archetypes: Controller
Allows execution of application specific actions that cannot be mapped with any
of the CRUD methods (specific to collections or repositories).
Should be avoided as much as possible. Actions should be reconsidered as one
of the proposed pseudo-archetypes: batch, queue, transaction.
This would allow execution to be more manageable and traceable.

44.
06.11.2015
44
Controller Archetype: URI & HTTP Verbs
• Base URI: http://api.domain.com/{version}/{param1}/{param2}/{controllerAction}
HTTP Verb Description
POST Executes the specific action, with the given parameters

45.
06.11.2015
45
API Archetypes: Queue
Allows execution of application specific actions (jobs) in an asynchronous way,
while allowing proper management and traceability of the action.
Each action, once queued, has an identifier and a status. Therefore each action
becomes a Resource.
Upon completion, the job document can embed the job result or can provide a
link to the actual result.

46.
06.11.2015
46
Queue Archetype: URI & HTTP Verbs
• Base URI: http://api.domain.com/{version}/{queue}/
HTTP Verb Description
GET /{id} Retrieves the queued job status, progress information and execution
result – if available.
POST Submits a new job into the queue and retrieves the job identifier.
DELETE /{id} Cancels a job (if the status allow the operation to be canceled)
OPTIONS Lists the available operations / HTTP Verbs

47.
06.11.2015
47
API Archetypes: Transaction
Allows execution of application specific actions in transactional manner.
Transactions can be asynchronous (see queues) or asynchronous.
In case of synchronous transactions, response contains the transaction result.

48.
06.11.2015
48
Transaction Archetype: URI & HTTP Verbs
• Base URI: http://api.domain.com/{version}/{queue}/
HTTP Verb Description
GET /{id} Retrieves the transaction status, progress information and execution
result – if available.
POST Submits a new transaction to be processed.
DELETE /{id} Stops/cancels a transaction (in case of asynchronous/queued
transactions)
OPTIONS Lists the available operations / HTTP Verbs

49.
06.11.2015
49
API Archetypes: Batch
Allows execution of application specific actions over multiple records / bulk
processing.
Batches must always be asynchronous, and the behavior is identical to Queues.
Progress information is mandatory in case of batches.

50.
06.11.2015
50
Batch Archetype: URI & HTTP Verbs
• Base URI: http://api.domain.com/{version}/{batch}/
HTTP Verb Description
GET /{id} Retrieves the status, progress information and execution result – if
available.
POST Submits a new batch job to be processed.
DELETE /{id} Stops/cancels a batch job
OPTIONS Lists the available operations / HTTP Verbs

51.
06.11.2015
51
Pseudo Archetypes
PRO TIP: You can combine Queues with
Transactions and Batches in any way you
want.
Just make sure things make sense 

52.
06.11.2015
52
Pseudo Archetypes
PRO TIP: Sometimes you might want to
have, for each Pseudo Archetype, a
validation equivalent.
Helps your applications/integrations cause
less frustrations.

53.
Error Handling
how do we handle errors &
failure?

54.
06.11.2015
54
API Error Handling
Use specific HTTP Error Codes where appropriate for application logic specific
errors. Ex:
- 401 Unauthorized: errors with authentication / authorization
- 404 Not Found: document does not exist
- ….

55.
06.11.2015
55
API Error Handling
Response body should always contain:
- Error code – what happened, coded for analytics
- Error message – human readable message, localized to user’s profile
- Error details – the context on the error.
Nice to have:
- Error ID – id of the error, generated by the lowest level module where the error
occurred. Should also propagate and find itself in the log files.
- Flow ID – if possible, for cross-app tracking

56.
06.11.2015
56
Pseudo Archetypes
PRO TIP: Certain operations can generate
multiple errors (ex. multiple validation
errors). If applicable, provide a list of errors
in the response instead of providing just the
first error encountered.

57.
06.11.2015
57
Pseudo Archetypes
PRO TIP: Don’t provide too much details in
the error messages. Consider privacy and
security before anything else.

58.
Testing
how do we make our API
bullet proof?

59.
06.11.2015
59
API Testing
Option 1:
- Develop API Client along the actual API Service
- Build Automated tests using the API Client, using the same programing
language

60.
06.11.2015
60
API Testing
Option 2:
- Build Automated tests using a dedicated API Testing framework
• http://frisbyjs.com/
• https://github.com/mkotsur/restito
• https://github.com/rest-driver/rest-driver
• https://github.com/dareid/chakram

61.
06.11.2015
61
API Testing
Option 3:
- Use dedicated 3rd Party apps or Browser Plugins
• https://www.getpostman.com/
• http://www.soapui.org/rest-testing/getting-started.html

62.
Security
how do we protect our data
(or ourselves)?

63.
06.11.2015
63
Securing Rest API
Simple rules:
- Use disposable authentication tokens
- Implement authentication mechanisms good for your target audience
Ex: OAuth2 works very well for web and mobile, but does it work for enterprise?
- Use granular roles & access rights for each resource model
- Take the multi-tenancy into account from start
- Use & require HTTPS whenever sensitive data is in transit
- Encrypt any sensitive data at rest

64.
06.11.2015
64
Securing Rest API
Preventive & Enterprise Ready Security:
- Audit any failed authentication / authorization
- When using sessions, ensure the expiration time can be configured
- Lock/disable authentication tokens after a configurable number of failed
authentication attempts
- Use a whitelist / blacklist for request filtering
- Implement overall rate limits as well as application / token specific limits
- Validate any data coming IN. Never take things for granted.

65.
06.11.2015
65
Securing Rest API
Preventive & Enterprise Ready Security:
- Require (configurable) password strength
- Use password expiration
- Log Remote IP, User-Agent (and Application ID, if possible) with any request
- Pass authentication tokens in the request headers, not in URL/Query String
- Filter unaccepted query string parameters from URL’s
- Escape/strip any textual information accepting user input

66.
06.11.2015
66
Pseudo Archetypes
PRO TIP: NEVER, EVER, EVER,
EVER
TRUST YOUR USERS.

67.
Manageability
how can we manage our API
and support it operation-
wise?

68.
06.11.2015
68
Managing Rest API & Supporting Operations
Simple rules:
- Log everything, almost everything.
- Implement usage analytics and metrics
- Time any operation, build performance metrics and alerts
- Build operational dashboards, enable error notifications
- Use change logs & audit trails for sensitive data
- Require “4 eyes check” rule on any major change performed by admins
- Use infrastructure as code

69.
Evolution
how do we ensure our API is
future-proof?

70.
06.11.2015
70
Future proof your API
Simple rules:
- Version your resource models and archetypes from the beginning
- Any new version of the resource model should extend the previous versions
- Establish a support plan from first public release
- Classify your API’s as public, private & unsupported to ensure phased
adoption

71.
Adoption
how do we make sure our
API will be adopted?

72.
06.11.2015
72
Ensuring APIs adoption
- Make it easy to get started - developers should be 1 CURL request away from
their first API call.
- Provide good and clear documentation.
- Provide SDK’s / client libraries and samples for most popular development
platforms and languages of your target audience.

73.
06.11.2015
73
Ensuring APIs adoption
- Ensure your business logic & rules don’t have to be guessed and duplicated by
developers in their own apps (ex. validation rules).
- Eat your own dog food. Use your API to build your other products /
integrations.
- Don’t introduce breaking changes. Plan to support your public API releases.

74.
06.11.2015
74
Ensuring APIs adoption
PRO TIP: Deliver VALUE,
not just JSON or XML.

75.
Thanks for watching. 