apidays LIVE Australia 2021 - Accelerating Digital September 15 & 16, 2021 API Horror Stories from an Unnamed Coworking Company Phil Sturgeon, DevRel at Stoplight

1. API Horror Stories
from an Unnamed
Coworking Company
@philsturgeon

2. API A ➡ API B ➡ API C ➡ API D ➡ API E ➡ ...
All entirely synchronously in web threads.

3. 2-5 seconds on a good day.
20-30 seconds on a bad day.

4. Overfetching and underfetching.

5. New York
JULY
Australia
SEPTEMBER
Singapore
APRIL
Helsinki & North
MARCH
Paris
DECEMBER
London
OCTOBER
Jakarta
FEBRUARY
Hong Kong
AUGUST
JUNE
India
MAY
Check out our API Conferences here
50+ events since 2012, 14 countries, 2,000+ speakers, 50,000+ attendees,
300k+ online community
Want to talk at one of our conferences?
Apply to speak here

6. No HTTP caching anywhere.

7. Different error formats emitted from the same
APIs, so customers seeing Error: [object Object]
was common.

8. Auth was enabled per-endpoint and disabled in
testing, so some sensitive production endpoints
had no authentication.

9. No APIs had any documentation at all.

10. No Time for Documentation.
Let's “Just” Rewrite It!

11. https://apisyouwonthate.com/blog/api-design-first-vs-code-first/

12. No client would be able to use newer versions of
the API, because:
a) designed for another clients requirements
b) no documentation anyway

13. Solutions
The API Design-first workflow.
Get a free account on stoplight.io to design, mock, and gather feedback early.
Those designs can be turned into beautiful docs with no extra effort.

14. Cache Stampede Cluedo

15. Cache Stampede

16. Only One Clue: “User-Agent: Faraday 0.9.1”

17. Cache Stampede Cluedo

18. App A Lockfile has… 0.9.0
App B Lockfile has… 0.8.1
App C Lockfile has… 0.9.3
…
App J Lockfile has… 0.9.1!
Cache Stampede Cluedo

19. Photo Credit: DenverData Web
https://www.glassdoor.co.uk/Photos/DenverData-Web-Office-Photos-IMG3315825.htm

20. Set User-Agent to “App Name {deploy version/hash}”
Setup Open Telemetry: opentelemetry.io to trace transactions.
Use a Service Mesh to specifically set which internal apps/APis can talk to what.
Service Mesh usually has tracing ready to go.
Solutions

21. Distributed Monoliths
Diagram Credit

22. “If you switch one of the microservices off
and anything else breaks, you don't really
have a microservice architecture, you just
have a distributed monolith!"
Domain Modelling Made Functional, by Scott Wlaschin

23. Diagram Credit

31. ��

32. ��
��
��

33. ��
��
��
��
��
��
��
��

34. ��
��
��
��
��
��
��
��
��
��
��
��
��
��
��

37. Graphic Credit
climatecare.org

38. Solutions
1. If you can’t design a good monolith with good separation of concerns, don’t
start adding network calls.
2. Create SLAs (Service Level Agreements) for your API and stick to them.
3. Set timeouts on every HTTP call, matching the SLA.
4. Expect to fail, then do something smart (queue, back off, hide a feature).
https://apisyouwonthate.com/blog/taking-a-timeout-from-poor-performance

40. Busy day in Australia?
Nobody poops!

46. ��

47. ��
��
��

48. ��
��
��
��
��
��
��
��
��

49. Them: “Our logging service doesn’t show
anything slower than 100ms”
Me: “I’ve enabled an outgoing traffic proxy
and requests that should work are randomly
taking >2min, then giving a 502.”
Them: “Oh that’s weird… nothing should ever
502. It should 200 on an error.”
Me: “Can we focus on the 2 minute part?!”
Them: “Oh yeah, that’s weird too...”
Some Other Company said it was our fault...

50. New Sync
Service
��
��

51. New
Async
Service
��
⏱

52. Solutions
1. Demand SLA for third-party services.
2. Pipe external traffic through a proxy like resurface.io or istio.io
3. Avoiding hitting APIs in a web thread whenever possible.
4. Especially if they’re not under your control.
5. Background Workers & Event Driven APIs for “things that can happen later”

53. Mutually Assured Destruction

55. {
"users": [
{
"id": "61373177894c0fd0d145a455",
"name": "Williamson Acosta",
"email": "williamsonacosta@xerex.com",
"picture": "http://placehold.it/32x32",
"age": 20,
"locale": "en-GB",
"phone": "+1 (932) 500-2370",
"registered": "2015-04-13T03:04:33 -01:00",
"locations": [
{
"id": 1,
"name": "Hunterfly Place",
"address": "642 Ridge Boulevard, Wattsville, Palau, 3038"
},
... snip 20 records ...
],
"companies": [
{
"id": 1,
"name": "Columella",
"address": "151 Stillwell Place, Cotopaxi, Texas, 7297",
"locations": [
{
"id": 1,
"name": "Alabama Avenue",
"address": "912 Madison Street, Marbury, Utah, 2225"
},
... snip 50 records ...
]
}
]
GET /v3/users/123
Size: 200kb
Response: 10-20s

56. A needs B needs A and A again

57. A needs B needs A and A again
⏱

58. A needs B needs A and A again
⏱
⏱

59. A needs B needs A and A again
��
⏱

60. A needs B needs A and A again
��
��

62. A asks B one small thing

63. 1. Stop designing for HTTP/1 (i.e. smashing everything in one mega call.)
2. Use HTTP/2 and HTTP/3 to multiplex multiple requests.
3. Clients only request more data if they want it - that’s what HTTP requests are.
4. Timeouts & Circuit Breakers so simple requests can succeed.
5. Get an API Architecture / Governance team to review changes.
Solutions
https://fastly.com/blog/optimise-api-cache-improved-performance

64. Stoplight.io can power your API Design & Governance workflows

65. Thank you for listening!
Got any Questions?
@philsturgeon

66. New York
JULY
Australia
SEPTEMBER
Singapore
APRIL
Helsinki & North
MARCH
Paris
DECEMBER
London
OCTOBER
Jakarta
FEBRUARY
Hong Kong
AUGUST
JUNE
India
MAY
Check out our API Conferences here
50+ events since 2012, 14 countries, 2,000+ speakers, 50,000+ attendees,
300k+ online community
Want to talk at one of our conferences?
Apply to speak here