Agility is enabled by systems that are flexible and extensible. IEEE Standard Glossary of Software Engineering Technology defines Flexibility as the ease with which a system or component can be modified for use in applications or environments other than those for which it was specifically designed. and Extensibility as the ease with which a system or component can be modified to increase its functional capacity. As customer needs evolve, flexibility helps with composing features and applications rapidly from using API, and extensibility helps in introducing the required changes quickly by making changes in services. The challenge many companies have today is that they have built systems primarily as monoliths during a stage of their evolution, or have several services but they are tightly coupled. Satisfying a customer need even though the functionality already exists can take significant effort and cost. Companies are therefore moving to APIs and microservices to address this business imperative. While many initiate this journey, most of them don’t end up with the intended outcomes - they falter or fail and go through many iterations. This talk will share principles, best practices and approaches - across architecture, organizational and cultural - that have been applied in transforming companies such as eBay, PayPal and Intuit to APIs and Microservices.

1. Deepak Nadig
Distinguished Architect,Intuit
@deepak_nadig
September 30,2016
Journey to APIs and Microservices:
Best Practices

2. 2

3. 3
Consumers Small Businesses Accounting Professionals
Who we serve:

4. 4
Where we started
Intuit’s journey began over 30 years ago when our founder Scott Cook sat at
his kitchen table and watched his wife as she balanced their checkbook and
thought there must be a better way.
Where we started:

5. 5
Era of the CloudEra of Windows Era of WebEra of DOS
2010+1980s 1990s 2000s
Proven track record: capitalizing on change
Intuit Founded
Quicken
DOS
IPO
Quicken
Window
s
QuickBooks
DOS
TurboTax Online
QuickBooks
Windows
TurboTax Windows
QuickBooks
MobileQuickBooks Online
QuickBooks
Online Global
• Employees: 150
• Customers: 1.3M
• Revenue: $33M
• INTU + 1,400%, NASDAQ + ~500%
• Employees: 4,500
• Customers: 5.6M
• Revenue: $1.04B
• INTU - 20%, NASDAQ - 40%
• Employees: 7,700
• Customers: 50M
• Revenue: $3.1B
• INTU +160%, NASDAQ 90%
INTUshareprice
SnapTax
Competitor
A
Competitor A Competitor C
Competitor A
Competitor A
Competitor B
Competitor
D
Competitor E

6. 6
Recognized as one of the world’s leading companies:
FORTUNE 100 BESTCOMPANIES TO WORK FOR - 15 Years in a Row
MOST ADMIRED: SOFTWARE INDUSTRY MOST INNOVATIVE COMPANIES
13 Years in a Row
2004 2005 2006 2007 2008 2009 2010 2011 2012 2015
WORLD’S MOST
ADMIRED
COMPANIES
2013 2014
2
Ranked
#
2016
2016
34
Ranked
#

7. 7
# 1 Small Business Management.7700
employees.$4.7 Billion in revenue.
QuickBooks has more than 5 million customers.
Online is 1.5 million (41% growth YoY).
26K developers. 2500 third-party apps.15% of
customers use at least one third-party app.
Yearly, 100 million invoices; $1.5 trillion in
commerce; payroll for more than 1 million
employees (1/12 in US).
Small Business Owners spend 4 hours online
every day.
QuickBooks is in 112 countries.
Intuit and QuickBooks by the numbers

8. 8
The capability of rapidly and cost efficiently adapting to changes
- Wikipedia
The ability of an organization to sense environment change and
to respond efficiently and effectively to it
- Gartner
Challenge for businesses today = Agility

9. 9
Architecture Business Cycle
organization (who, together, fashioned the quality-attribute requirements), the technical
environment, and the architect’s experience.
In our adaptation of the ABC, the “influences” have been replaced by a set of forces that
collectively shape the requirements and, over time, the architect’s experience. The
extended ABC is depicted in Figure 1.
Figure 1: Extending the Architecture Business Cycle to Include All Design Forces
Notice that this extended ABC shows the requirements as being categorized into three
overlapping sets: quality-attribute requirements (e.g., the system should initialize itself
in 60 seconds), business requirements (e.g., version 1.0 must be available to our
resellers by August 1), and functional requirements (e.g., the system must provide
monthly sales summaries, broken down by region). These requirements, along with the
Software Engineering Institute http://sei.cmu.edu/
Architecture Business Cycle has impact on Architecture, Business and Culture

10. 10
• Architecture journey
• Business journey
• Culture journey
• Current à Target ABC
S-Curve
Business
Performance
Time
Kickoff
High
Growth
Limits
Reached
ABC
ABC
ABC
(*) ABC = Architecture Business Cycle

11. 11
Flexibility and Extensibility
Flexibility
Ease with which a system or component can be
modifiedfor usein applicationsor environmentsother
than thosefor which it was specificallydesigned
Extensibility
Ease with which a system or component can be
modifiedto increaseits functionalcapacity
IEEE Standard Glossary of Software Engineering Technology

12. 12
Agility is enabled by Flexibility and Extensibility
APIs enable flexibility
Microservices enable extensibility
Why APIs and Microservices?

13. 13
Journey begins as one of the following …
UI and
Business Logic
Web Browser
Web UI
Web Browser
Business Logic
Web UI
Web
Browser
Business Logic
Mobile UI
Mobile
Web UI
Web
Browser
Business
Logic
Mobile UI
Mobile
We need microservices!
Why are we still slow?
We need mobile!
No time to refactor
Develop feature N times
Just wantto tweak UI!
How do we organize?
Yay! Growingfast
One language
Slowing releases

14. 14
Target State
API Definition Internal or External Universal
API Discovery Painful Developer Portal
API Design Project specific API as a Product
Architecture Tightly coupled SOA Loosely coupled SOA
Technology Proprietary Standards based
Integration Expensive TTFHW* < 5 min
(*) TTFHW = Time To First Hello World

15. 15
Portfolio of APIs
aligned by business capabilities,
realized by isolated and encapsulated services,
that can be used by internal and external developers
to develop applicationsand integrations
quicklyand cost effectively
API Platform

16. 16
API Platform Principles
API First
API Design
• Think API before experience
• API is a product
• Externalize-able
• Domain model & Integration scenarios
Developer Experience
• Easy to learn, integrate, diagnose
• Be the ‘developer’
API Quality Attributes
• Response-time
• Availability
Service Architecture
• Encapsulated, Isolated
• Craftsmanship

17. 17
• Know the end user
• Work back from the developer and integration
• Treat 1st party = 2nd party = 3rd party
• Make it easy to integrate
• Zero impact to integration as API evolves
• Loosely coupled services
Elements in building an API Platform

18. Best Practices
API as a product
API design
Experience vs Capability API
API specification
API documentation
Service granularity
Service development
API testing
API status & support
Developer Portal

19. 19
End users
Understand Segments
Their Experiences
And Expectations
Developers
Know Tools and processes
Their Integration needs
Be Transparent on usage
API as a Product

20. 20
API Portfolio
Define bounded contexts/capabilities
Organize cohesive concepts
Use orthogonality or distance to separate
API Design
Use problem space for domain model
Consider all interactions – Events, Batch
Plan to dog food API (and events)
Share designs (apistylebook.com)
API Design
id
id
/invoicing /payments
/risk
DomainModel à API
• Entities, Attributes
• Verbs
• Relationships
• State machine

21. 21
Capability API
What API teams deliver
Optimizedfor API provider
REST, Batch, …
Experience API
Personalized for end user experience
Make it self-service for developer
Plan for multiple- GraphQL, Groovy
Experience vs Capability API
API Facade
Invoicing Payments Customer
Experience API
Capability API

22. 22
Many choices
OAS*, JSON Schema
Capability,Experience API
Automatic Docs, SDKgeneration
Maintain as code
GitHub, GitFlow(agile versioning)
Pull Request (Governance)
Share it
API Specification
(*) Open API Specification

23. 23
By developers for developers
Developers contribute documentation
Technical writers are editors
Include in release criteria
Maintain as code
GitHub, GitFlow
Should be tested like code
Open it for contributions
API Documentation

24. 24
Why you should care?
Extensibility
Granularity Principles
Team boundary
Cohesion
Security
Rate of change
Scalability
Service Granularity
Credit: Spotify Engineering Culture

25. 25
Getting to microservices
Code complexity vs. change
Pick the first, complete, repeat
Team autonomy, diagnosingissues
New microservice
Plan for multiple languages
Plan for CI
Team (4) > Team (2) (45% more time)*
Service Development
Complexity
Low
High
High
Low
Least priority
Low priority
Impacting cycle
time
Easier to
decompose
Start here
Impacting cycle
time most
Decompose,
then refactor
Change
(*) The team scaling fallacy: Underestimating the declining efficiency of larger teams

26. 26
API Testing
Test independently
API specificationcanhelp bootstrap
Complete scenarios (e.g. Webhooks)
Sandbox
Sandbox = Production
For internal and external developers
Support automatedtesting
API Testing

27. 27
API Status
Developers need to know
Healthdashboard
Report status promptly on issues
Diagnosing issues
Treat developers alike
Provide API logs with messages
Be proactive using API analytics
API Status & Support

28. 28
Complete developer experience
For 1st and 3rd party developers
Time To First Hello World < 5 min
Developer Portal

29. Culture

30. 30
A way of thinking,behaving, or working
that exists in a place or organization
- Merriam-Webster
Behavior of humanswithin an organization and
the meaning that people attach to those behaviors.
- Wikipedia
What is culture?

31. 31
Application development
Web page design Process
Site design/Portfolio Management
Engineers work with page designs
API development
API DesignProcess
API Portfolio Management
Engineers work with API designs
API Engineering Processes
App. Product
Manager
UED*
Application
Engineering
API Product
Manager
API
Designer
Service
Engineering
(*) UED = User Experience Design

32. 32
Be aware of Conway’s law
Namespace != Organization
Reuse = Coupling
Agreements based on standards
Teams evolve at different rates
Use maturity model to measure
Organization Evolution
How do committees invent?

33. 33
Build something in 24 hours
Internal hackathons
Usability testing for APIs
Pride of ownership for API teams
Developer advocacy
Hackathons

34. 34
• Agility is a business imperative
• Agility comes from flexible and extensible
architectures
• APIs enable flexibility
• Microservices enable extensibility
• Architecture and culture evolve together
Summary

35. Thank You