Specification-Driven Development of REST APIs, Alexander Zinchuk, Software Architect at Toptal

1. Alexander Zinchuk
SPECIFICATION-DRIVEN
DEVELOPMENT
ajaxy_ru

2. Alexander Zinchuk
toptal.com/resume/
alexander-zinchuk
anywaylabs.com
Executive EngineerSoftware Architect
ajaxy_ru

3. Alexander Zinchuk
SPECIFICATION-DRIVEN
DEVELOPMENT
ajaxy_ru

4. WHAT IS A
SPECIFICATION?
SPECIFICATION-DRIVEN DEVELOPMENT
ajaxy_ru

5. Industry standard
for REST API spec:
(FORMER SWAGGER)
SPECIFICATION-DRIVEN DEVELOPMENT
ajaxy_ru

6. MAINTAINING OPENAPI SPEC
{
"swagger": "2.0",
"info": {
"title": "Flightcall API 2.0",
"description": "This document describes HTTP REST JSON API",
"version": "2.0.0"
},
"host": "api.flightcall.flightvector.com",
"basePath": "/",
"schemes": [
"https"
],
"consumes": [
"application/x-www-form-urlencoded"
],
"produces": [
"application/json"
],
"securityDefinitions": {
"token": {
"name": "Authorization",
"type": "apiKey",
"in": "header"
}
},
"paths": {
"/organizations": {
"get": {
"summary": "List all organizations",
"description": "List all organizations",
"operationId": "GET--organizations",
"responses": {
"200": {
"description": "",
"schema": {
"type": "array",
"items": {
"$ref": "#/definitions/Organization"
}
}
}
},
"tags": [
"Public endpoints"
]
}
},
"/organizations/{id}/ems_agencies": {
"get": {
"summary": "List all EMS agencies tied to a specific organization",
"description": "List all EMS agencies tied to a specific organization",
"operationId": "GET--organizations--id--ems_agencies",
"responses": {
"200": {
"description": "",
"schema": {
"type": "array",
"items": {
"$ref": "#/definitions/EmsAgency"
}
}
}
...
openapi.json
ajaxy_ru

7. MAINTAINING OPENAPI SPEC
{
"swagger": "2.0",
"info": {
"title": "Flightcall API 2.0",
"description": "This document describes HTTP REST JSON API",
"version": "2.0.0"
},
"host": "api.flightcall.flightvector.com",
"basePath": "/",
"schemes": [
"https"
],
"consumes": [
"application/x-www-form-urlencoded"
],
"produces": [
"application/json"
],
"securityDefinitions": {
"token": {
"name": "Authorization",
"type": "apiKey",
"in": "header"
}
},
"paths": {
"/organizations": {
"get": {
"summary": "List all organizations",
"description": "List all organizations",
"operationId": "GET--organizations",
"responses": {
"200": {
"description": "",
"schema": {
"type": "array",
"items": {
"$ref": "#/definitions/Organization"
}
}
}
},
"tags": [
"Public endpoints"
]
}
},
"/organizations/{id}/ems_agencies": {
"get": {
"summary": "List all EMS agencies tied to a specific organization",
"description": "List all EMS agencies tied to a specific organization",
"operationId": "GET--organizations--id--ems_agencies",
"responses": {
"200": {
"description": "",
"schema": {
"type": "array",
"items": {
"$ref": "#/definitions/EmsAgency"
}
}
}
...
openapi.json
ajaxy_ru

8. MAINTAINING OPENAPI SPEC
{
"swagger": "2.0",
"info": {
"title": "Flightcall API 2.0",
"description": "This document describes HTTP REST JSON API",
"version": "2.0.0"
},
"host": "api.flightcall.flightvector.com",
"basePath": "/",
"schemes": [
"https"
],
"consumes": [
"application/x-www-form-urlencoded"
],
"produces": [
"application/json"
],
"securityDefinitions": {
"token": {
"name": "Authorization",
"type": "apiKey",
"in": "header"
}
},
"paths": {
"/organizations": {
"get": {
"summary": "List all organizations",
"description": "List all organizations",
"operationId": "GET--organizations",
"responses": {
"200": {
"description": "",
"schema": {
"type": "array",
"items": {
"$ref": "#/definitions/Organization"
}
}
}
},
"tags": [
"Public endpoints"
]
}
},
"/organizations/{id}/ems_agencies": {
"get": {
"summary": "List all EMS agencies tied to a specific organization",
"description": "List all EMS agencies tied to a specific organization",
"operationId": "GET--organizations--id--ems_agencies",
"responses": {
"200": {
"description": "",
"schema": {
"type": "array",
"items": {
"$ref": "#/definitions/EmsAgency"
}
}
}
...
openapi.json
VERBOSE AND BORING
ajaxy_ru

9. Attempts to optimize:
· Multiple files
· JSDoc
· Online editors and services
MAINTAINING OPENAPI SPEC
ajaxy_ru

10. Augmenting this presentation with
examples
MAINTAINING OPENAPI SPEC
TINYSPEC
ajaxy_ru

11. MAINTAINING OPENAPI SPEC
Imagine, we need to
Get users…
ajaxy_ru

12. User {name, age?: i, isAdmin: b}
user.models.tinyspec
MAINTAINING OPENAPI SPEC
ajaxy_ru
Imagine, we need to
Get users…

13. User {name, age?: i, isAdmin: b}
user.models.tinyspec
GET /users
=> {users: User[]}
users.endpoints.tinyspec
MAINTAINING OPENAPI SPEC
ajaxy_ru
Imagine, we need to
Get users…

14. npmjs.com/package/tinyspec
MAINTAINING OPENAPI SPEC
ajaxy_ru

15. User {name, age?: i, isAdmin: b}
GET /users
=> {users: User[]}
user.models.tinyspec
users.endpoints.tinyspec
{
"swagger": "2.0",
"info": {
"title": "API Example",
"description": "API Example",
"version": "1.0.0"
},
"paths": {
"/users": {
"get": {
"summary": "GET /users",
"description": "GET /users",
"operationId": "GET--users",
"responses": {
"200": {
"description": "",
"schema": {
"type": "object",
"properties": {
"users": {
"type": "array",
"items": {
"$ref": "#/definitions/User"
}
}
},
"required": [
"users"
]
}
}
}
}
}
},
"tags": [],
"definitions": {
"User": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"age": {
"type": "integer"
},
"isAdmin": {
"type": "boolean"
}
},
"required": [
"name",
"isAdmin"
]
}
}
}
openapi.json
$ tinyspec -–json
MAINTAINING OPENAPI SPEC
ajaxy_ru
Imagine, we need to
Get users…

16. MAINTAINING OPENAPI SPEC
.org
ajaxy_ru

17. YOU CAN RE-USE
SPEC IN CODE!
SPECIFICATION-DRIVEN DEVELOPMENT
ajaxy_ru

18. 1
ENDPOINT UNIT TESTS
ajaxy_ru

19. npmjs.com/package/supertest rubygems.org/gems/airborne
npmjs.com/package/chai-http
1 · ENDPOINT UNIT TESTS
ajaxy_ru

20. 1 · ENDPOINT UNIT TESTS
describe('/users', () => {
it('List all users', async () => {
const { status, body: { users } } = request.get('/users');
expect(users[0].name).to.be('string');
expect(users[0].isAdmin).to.be('boolean');
expect(users[0].age).to.be.oneOf(['boolean', null]);
});
});
describe 'GET /users' do
it 'List all users' do
get '/users'
expect_json_types('users.*', {
name: :string,
isAdmin: :boolean,
age: :integer_or_null,
})
end
end
npmjs.com/package/supertest rubygems.org/gems/airborne

21. User {name, age?: i, isAdmin: b}
GET /users
=> {users: User[]}
user.models.tinyspec
users.endpoints.tinyspec
{
"swagger": "2.0",
"info": {
"title": "API Example",
"description": "API Example",
"version": "1.0.0"
},
"paths": {
"/users": {
"get": {
"summary": "GET /users",
"description": "GET /users",
"operationId": "GET--users",
"responses": {
"200": {
"description": "",
"schema": {
"type": "object",
"properties": {
"users": {
"type": "array",
"items": {
"$ref": "#/definitions/User"
}
}
},
"required": [
"users"
]
}
}
}
}
}
},
"tags": [],
"definitions": {
"User": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"age": {
"type": "integer"
},
"isAdmin": {
"type": "boolean"
}
},
"required": [
"name",
"isAdmin"
]
}
}
}
openapi.json
$ tinyspec -–json
1 · ENDPOINT UNIT TESTS
ajaxy_ru
Imagine, we need to
Get users…

22. 1 · ENDPOINT UNIT TESTS
json-schema.org
"User": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"age": {
"type": "integer"
},
"isAdmin": {
"type": "boolean"
}
},
"required": [
"name",
"isAdmin"
]
}
JSON SCHEMA
ajaxy_ru

23. 1 · ENDPOINT UNIT TESTS
Any key-value object may be validated against JSON Schema
json-schema.org
"User": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"age": {
"type": "integer"
},
"isAdmin": {
"type": "boolean"
}
},
"required": [
"name",
"isAdmin"
]
}
JSON SCHEMA
ajaxy_ru

24. npmjs.com/package/jest-ajv rubygems.org/gems/json_matchers
npmjs.com/package/chai-ajv-json-schema
1 · ENDPOINT UNIT TESTS
Any key-value object may be validated against JSON Schema
ajaxy_ru

25. 1 · ENDPOINT UNIT TESTS
import deref from 'json-schema-deref-sync';
const schemas = deref(require('./openapi.json')).definitions;
describe('/users', () => {
it('List all users', async () => {
const { status, body: { users } } = request.get('/users');
expect(users[0]).toMatchSchema(schemas.User);
});
});
require ‘json_matchers/rspec'
JsonMatchers.schema_root = 'spec/schemas'
describe 'GET /users' do
it 'List all users' do
get '/users'
expect(result[:users][0]).to match_json_schema('User')
end
end
npmjs.com/package/jest-ajv rubygems.org/gems/json_matchers

26. 2
USER-DATA VALIDATION
ajaxy_ru

27. 2 · USER-DATA VALIDATION
User {name, age?: i, isAdmin: b}
user.models.tinyspec
Imagine, we need to
Update a user…
ajaxy_ru

28. 2 · USER-DATA VALIDATION
User {name, age?: i, isAdmin: b}
UserUpdate !{name?, age?: i}
user.models.tinyspec
Imagine, we need to
Update a user…
ajaxy_ru

29. 2 · USER-DATA VALIDATION
User {name, age?: i, isAdmin: b}
UserUpdate !{name?, age?: i}
user.models.tinyspec
Imagine, we need to
Update a user…
ajaxy_ru

30. 2 · USER-DATA VALIDATION
User {name, age?: i, isAdmin: b}
UserUpdate !{name?, age?: i}
user.models.tinyspec
Imagine, we need to
Update a user…
ajaxy_ru

31. PATCH /users/:id {user: UserUpdate}
=> {success: b}
users.endpoints.tinyspec
2 · USER-DATA VALIDATION
UserUpdate !{name?, age?: i}
user.models.tinyspec
Imagine, we need to
Update a user…
ajaxy_ru

32. UserUpdate !{name?, age?: i}
PATCH /users/:id {user: UserUpdate}
=> {success: b}
user.models.tinyspec
users.endpoints.tinyspec
2 · USER-DATA VALIDATION
...
"paths": {
"/users/{id}": {
"patch": {
"summary": "PATCH /users/:id",
"description": "PATCH /users/:id",
"operationId": "PATCH--users--id",
"responses": {
"200": {
"description": "",
"schema": {
"type": "object",
"properties": {
"success": {
"type": "boolean"
}
},
"required": [
"success"
]
}
}
},
"parameters": [
{
"name": "id",
"type": "string",
"in": "path",
"required": true
},
{
"name": "body",
"required": true,
"schema": {
"type": "object",
"properties": {
"user": {
"$ref": "#/definitions/UserUpdate"
}
},
"required": [
"user"
]
},
"in": "body"
}
]
}
}
},
"tags": [],
"definitions": {
"UserUpdate": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"age": {
"type": "integer"
}
},
"additionalProperties": false
}
}
openapi.json
Imagine, we need to
Update a user…
$ tinyspec -–json
ajaxy_ru

33. npmjs.com/package/ajv rubygems.org/gems/json-schema
2 · USER-DATA VALIDATION
ajaxy_ru

34. router.patch('/:id', async (ctx) => {
const updateData = ctx.body.user;
// Validation using JSON schema from API specification.
await validate(schemas.UserUpdate, updateData);
const user = await User.findById(ctx.params.id);
await user.update(updateData);
ctx.body = { success: true };
});
2 · USER-DATA VALIDATION
ajaxy_ru

35. router.patch('/:id', async (ctx) => {
const updateData = ctx.body.user;
// Validation using JSON schema from API specification.
await validate(schemas.UserUpdate, updateData);
const user = await User.findById(ctx.params.id);
await user.update(updateData);
ctx.body = { success: true };
});
2 · USER-DATA VALIDATION
FieldsValidationError {
error: b,
message,
fields: {name, message}[]
}
error.models.tinyspec
ajaxy_ru

36. router.patch('/:id', async (ctx) => {
const updateData = ctx.body.user;
// Validation using JSON schema from API specification.
await validate(schemas.UserUpdate, updateData);
const user = await User.findById(ctx.params.id);
await user.update(updateData);
ctx.body = { success: true };
});
2 · USER-DATA VALIDATION
FieldsValidationError {
error: b,
message,
fields: {name, message}[]
}
error.models.tinyspec
PATCH /users/:id {user: UserUpdate}
=> 200 {success: b}
=> 422 FieldsValidationError
users.endpoints.tinyspec
ajaxy_ru

37. 3
MODEL SERIALIZATION
ajaxy_ru

38. 3 · MODEL SERIALIZATION
{…}
DB TABLE JSON VIEWORM MODEL
ajaxy_ru

39. {…}
ORM MODEL JSON VIEW
serialization
DB TABLE
3 · MODEL SERIALIZATION
ajaxy_ru

40. {…}
{…}
{…}
DB TABLE
ASSOCIATED
MODELS ALTERNATE
REQUESTS
3 · MODEL SERIALIZATION
ajaxy_ru

41. Our Spec
has all needed
information!
3 · MODEL SERIALIZATION
{…}
{…}
{…}
ALTERNATE
REQUESTSajaxy_ru

42. Imagine, we need to
Get all users
with posts
and comments…
3 · MODEL SERIALIZATION
ajaxy_ru

43. Comment {authorId: i, message}
Post {topic, message, comments?: Comment[]}
User {name, isAdmin: b, age?: i}
UserWithPosts < User {posts: Post[]}
models.tinyspec
3 · MODEL SERIALIZATION
ajaxy_ru

44. Comment {authorId: i, message}
Post {topic, message, comments?: Comment[]}
User {name, isAdmin: b, age?: i}
UserWithPosts < User {posts: Post[]}
GET /blog/users
=> {users: UserWithPosts[]}
models.tinyspec
blogUsers.endpoints.tinyspec
3 · MODEL SERIALIZATION
ajaxy_ru

45. Comment {authorId: i, message}
Post {topic, message, comments?: Comment[]}
User {name, isAdmin: b, age?: i}
UserWithPosts < User {posts: Post[]}
GET /blog/users
=> {users: UserWithPosts[]}
models.tinyspec
blogUsers.endpoints.tinyspec
3 · MODEL SERIALIZATION
…
»paths»: {
"/blog/users": {
"get": {
"summary": "GET /blog/users",
"description": "GET /blog/users",
"operationId": "GET--blog--users",
"responses": {
"200": {
"description": "",
"schema": {
"type": "object",
"properties": {
"users": {
"type": "array",
"items": {
"$ref": "#/definitions/UserWithPosts"
}
}
},
"required": [
"users"
]
}
}
}
}
}
},
"tags": [],
"definitions": {
"Comment": {
"type": "object",
"properties": {
"authorId": {
"type": "integer"
},
"message": {
"type": "string"
}
},
"required": [
"authorId",
"message"
]
},
"Post": {
"type": "object",
"properties": {
"topic": {
"type": "string"
},
"message": {
"type": "string"
},
"comments": {
"type": "array",
"items": {
"$ref": "#/definitions/Comment"
}
}
},
"required": [
"topic",
"message"
]
},
"User": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"isAdmin": {
"type": "boolean"
},
"age": {
"type": "integer"
}
},
"required": [
"name",
"isAdmin"
]
},
"UserWithPosts": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"isAdmin": {
"type": "boolean"
},
"age": {
"type": "integer"
},
"posts": {
"type": "array",
"items": {
"$ref": "#/definitions/Post"
}
}
},
"required": [
"name",
"isAdmin",
"posts"
]
}
}
openapi.json
$ tinyspec -–json
ajaxy_ru

46. 3 · MODEL SERIALIZATION
npmjs.com/package/sequelize-serialize
ajaxy_ru

47. 3 · MODEL SERIALIZATION
import serialize from 'sequelize-serialize';
router.get('/blog/users', async (ctx) => {
const users = await User.findAll({
include: [{
association: User.posts,
include: [
Post.comments
]
}]
});
ctx.body = serialize(users, schemas.UserWithPosts);
});
npmjs.com/package/sequelize-serialize
ajaxy_ru

48. -=MAGIC=-

49. 4
STATIC TYPING
ajaxy_ru

50. 4 · STATIC TYPING
JSON SCHEMA
ajaxy_ru

51. npmjs.com/package/sw2dts
npmjs.com/package/swagger-to-flowtype
4 · STATIC TYPING
ajaxy_ru

52. 4 · STATIC TYPING
$ tinyspec -–json
$ sw2dts ./openapi.json -o Api.d.ts --namespace Api
ajaxy_ru

53. 4 · STATIC TYPING
$ tinyspec -–json
$ sw2dts ./openapi.json -o Api.d.ts --namespace Api
declare namespace Api {
export interface Comment {
authorId: number;
message: string;
}
export interface Post {
topic: string;
message: string;
comments?: Comment[];
}
export interface User {
name: string;
isAdmin: boolean;
age?: number;
}
export interface UserUpdate {
name?: string;
age?: number;
}
export interface UserWithPosts {
name: string;
isAdmin: boolean;
age?: number;
posts: Post[];
}
}
Api.d.ts

54. router.patch('/users/:id', async (ctx) => {
// Specify type for request data object
const userData: Api.UserUpdate = ctx.request.body.user;
// Run spec validation
await validate(schemas.UserUpdate, userData);
// Query the database
const user = await User.findById(ctx.params.id);
await user.update(userData);
// Return serialized result
const serialized: Api.User = serialize(user, schemas.User);
ctx.body = { user: serialized };
});
4 · STATIC TYPING
ajaxy_ru

55. it('Update user', async () => {
// Static check for test input data.
const updateData: Api.UserUpdate = { name: MODIFIED };
const res = await request.patch('/users/1', { user: updateData });
// Type helper for request response:
const user: Api.User = res.body.user;
expect(user).to.be.validWithSchema(schemas.User);
expect(user).to.containSubset(updateData);
});
4 · STATIC TYPING
router.patch('/users/:id', async (ctx) => {
// Specify type for request data object
const userData: Api.UserUpdate = ctx.request.body.user;
// Run spec validation
await validate(schemas.UserUpdate, userData);
// Query the database
const user = await User.findById(ctx.params.id);
await user.update(userData);
// Return serialized result
const serialized: Api.User = serialize(user, schemas.User);
ctx.body = { user: serialized };
});
ajaxy_ru

56. 5
TYPE CASTING
ajaxy_ru

57. 5 · TYPE CASTING
param1=value&param2=777&param3=false
Query params or non-JSON body:
ajaxy_ru

58. 5 · TYPE CASTING
param1=value&param2=777&param3=false
{
param1: 'value',
param2: '777',
param3: 'false'
}
Query params or non-JSON body:
ajaxy_ru

59. npmjs.com/package/cast-with-schema
5 · TYPE CASTING
ajaxy_ru

60. import castWithSchema from 'cast-with-schema';
router.get('/posts', async (ctx) => {
// Cast parameters to expected types
const query = castWithSchema(ctx.query, schemas.PostsQuery);
// Run spec validation
await validate(schemas.PostsQuery, query);
// Query the database
const posts = await Post.search(query);
// Return serialized result
ctx.body = { posts: serialize(posts, schemas.Post) };
});
npmjs.com/package/cast-with-schema
5 · TYPE CASTING
ajaxy_ru

61. THANK YOU!
ajaxy_ru
github.com/Ajaxy/tinyspec anywaylabs.com