Public API
•
0 gostou•76 visualizações
Conference talk: Will the Real Public API Please Stand Up Demos here: https://github.com/azuker/public-api-demos
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Semelhante a Public API
Semelhante a Public API (20)
Último
Último (20)
Public API
- 1. Will the Real Public API Please Stand Up Amir Zuker Senior Software Architect Founder, Head of Web and Mobile Twitter: @AmirZuker
- 2. Fancy Books
- 5. Fancy Books MVP Orders Inventory Exploration Augmentation CDN Domain LB / API Gateway http websocket
- 6. Fancy Books MVP Orders Inventory Exploration Augmentation CDN Domain LB / API Gateway http websocket Of course, we might use additional services as needed.. queues, pub/sub, blobs, notifications, socket-at-scale, keys/secrets telemetry, monitoring, logs, analytics, etc.
- 10. Complexities start to emerge..
- 11. Fancy Books Orders Inventory Exploration Augmentation CDN Domain LB / API Gateway http websocket
- 13. Prelude Wrap-up So.. we don't want to be there This talk doesn't focus about HTTP RESTful API conventions How can we do it better then? 17
- 14. About Me 18 Amir Zuker Mentor, leader, architect, developer, instructor, author and speaker Over 20 years of experience in the field Leads teams and R&D divisions Several publications and contributions Head of Web and Mobile at CodeValue
- 15. Agenda Public API as a Product Beyond API Features Subscriptions and Identity Spec Authoring Versioning Design Philosophy 19
- 16. Public API as a Product 20
- 17. API exists in every code we write
- 18. Our focus: distributed API's
- 19. 23 Distributed API's – Protocols and Paradigms Many things to choose from TCP / UDP / HTTP / AMQP / MQTT / WebSocket / STOMP RESTful, HATEOS, gRPC, GraphQL, OData, AsyncAPI Depending on your needs
- 20. How much public is public?
- 21. Public API Product DESIGN DOCS COMM PORTALS VERSIONING TESTING RELEASES MONITOR SLA SUPPORT
- 22. Build What You Need, and Build It Well Clarify requirements Understand the use-cases involved Unclear? perhaps postpone it or roll it out as an alpha version Creativity and generosity can be an Achilles heel KISS Build good API’s 26
- 23. API Design is UX for Developers
- 24. Good API’s Why? 28 Adoption Competitiveness Maintainability Support Cost Manageability
- 26. Beyond API Features Routing and load balancing Discovery IP access listing Audit, analytics and billing Rate-limiting Content transformation Aggregations Security, PEP (e.g., sanitation) Telemetry and logging Versioning Caching Circuit breaker Protocol bridging Subscription and AUTH
- 27. Beyond API Features Many things to consider and take care of Can elevate concerns from the application code Leverage existing tools, platforms and services, e.g., APIM and API Gateways Service Mesh API Gateway and Service Mesh are not the same Nor is it mutually exclusive SaaS / OTS / OSS / DIY 31
- 28. API Management (APIM) 32 Client Client Client API Gateway Routing Security Audit Transform Log Inventory Service Orders Service Inventory Service Inventory Service Orders Service Orders Service
- 29. Edge / Internal / Graph Can be complex Multi-purpose components Edge / ingress Horizontal integrations BFF's Combine several tools Depends on your needs 33 Source
- 30. Many Tools Exist in the Wild 35 AWS API Gateway Azure APIM GCP API Gateway
- 33. Subscriptions Consumers Register client apps Retrieve credentials Sandbox integration Consume API's BackOffice Subscription portal 38
- 34. Identity and Auth What type of integration is needed? Need to identify individual identities and possibly SSO? Identity federation or synchronization Tenant trust Etc. What type of protocol and credentials? Basic, OAuth, OAuth2, OpenID, OpenID Connect Id + Secret / Tokens (e.g. jwt) / Certificates / API keys 39
- 35. API Keys + Token Exchange 40 Client Client Client API Gateway Cache Inventory Service Orders Service Inventory Service Inventory Service Orders Service Orders Service GET http://host:port/api/v1/books/0?apiKey=30c050f7fdfe466b9442570ae77dbc8a X-API-KEY: 30c050f7fdfe466b9442570ae77dbc8a ... IdM GET http://internal:port/api/books/0 Authorization: Bearer asdaSXAD2dsfsdVSDFSDsad.. ...
- 38. Contract First vs. Code First 43 Source
- 39. Contract First vs. Code First 44 Source Source
- 40. Contract First vs. Code First 45 Source Source
- 41. Contract First vs. Code First 46 Source Source Middle Ground
- 42. Auto Generation Auto generate as much as possible Tests and validators Changes and release notes Implementation (contract first) Client-side projects and SDK's Might require additional work Can adjust to specific needs 49
- 43. Documentation API lives or dies by its docs Document your API’s API surface General concerns Etc. API Explorer SaaS / OTS / OSS / DIY
- 45. Versioning 52
- 46. Versioning Concerns Commitment Docs and release notes Communication Tests, QA and releases Processes and workflows Environment versions Breaking changes System's multi-version support Codebase and conventions From a request to a version 53
- 47. Semantic Versioning Major Often include breaking changes Minor Expansion of the previous version Non-breaking changes only Patch Internal changes or fixes No new definitions No removal of definitions 54 Source
- 48. Breaking Changes Know what breaks Not just the contract, behavior and SLA too Test and validate Prefer expansion 55 Source
- 49. Commitment Decide what to commit to Long-term / short-term Example: New major every 2 years, 1 year side-by-side, 6 months beta Commit to 2 major versions at any point in time 56 2021 2022 2023 2024 2025 2026 2027 2028 GA v1 EoL GA EoL v2 beta dev GA EoL beta dev v3 GA beta dev v4
- 50. Multiple Versions How to support multiple versions? Side-by-side (major only?) Routing + transformation Code + transformation Hybrid How to correlate a request to a version? URL (host, path, query) Headers (standard, custom) Media type Mixed 57 GET /books Host: api.fancybooks.com Accept: application/json X-API-Version: 2 GET /v1/books Host: api.fancybooks.com Accept: application/json GET /books?v=2020-05-27 Host: api.fancybooks.com Accept: application/json GET /books Host: apiv1.fancybooks.com Accept: application/json GET /books Host: api.fancybooks.com Accept: application/vnd.fbooks.v1+json
- 53. Design Philosophy 60 Innovative vs. Conventional
- 54. Design Philosophy 61 Abundant vs. Minimalistic
- 55. Design Philosophy 62 Flexible vs. Opinionated
- 56. Design Philosophy 63 Clear vs. Vague
- 57. Design Philosophy 64 Performance vs. Simplicity
- 58. Design Philosophy 65 internal: Strict vs. Permissive
- 59. Regardless, Generally.. Be stable Be consistent Be symmetric Be informative where you’re willing to be Hide implementation details 66
- 60. API Surface Feature-based functionality Understand needs and underlying use-cases Factor in technical concerns as well Performance Latency Idempotence Concurrency Security Etc. 67
- 61. Recap 68
- 62. Recap Public API as a product Beyond API Features Subscriptions and Identity Spec Authoring Versioning Design Philosophy 69
- 63. Public API is a Product In every aspect.. Processes and workflows Tools, platforms and technologies Pipelines, tests, releases Can do agility first, maturity later Different lengths, mileage may vary 70 APIM Lifecycle Define Design Version Mock Develop Test Deploy Publish Monitor Engage Monetize
- 64. Key Takeaways There are many concerns to take care of Subscriptions, identities, security, observability, analytics, billing, etc. There are many tools, technologies, platforms and services to choose from Choose what coexists well with your platform API gateways and service mesh can go a long way Documentation is key Versioning is a big topic Make decisions Processes, design, coding, etc. 71
- 67. Amir Zuker Senior Software Architect Founder, Head of Web and Mobile Twitter: @AmirZuker
- 68. Q A 75
Notas do Editor
- Thank you Maayan for the warm welcome, Hello everyone, and welcome to my talk about public API’s My name is Amir Zuker, and let’s get going
- Let's say we're a book store company Fancy Books! We sell books, but have additional cool features We augment books, add narration and visualizations, and we integrate ML/AI processes and pipelines to make it happen. MVP - We need to build a simple system, webapp + backend We go with a cloud-based system, because, well.. why not
- Ok, so we have our backend, few deployable units, nice Modular monolith, serverless, containerized workloads (macro/micro/nano svcs) – doesn’t matter
- We need Web FE – CDN, Domain, browsers connected, cool
- We add additional services to bind everything together.. Cool, now it works
- New features and changes need to roll out We change the code We control the entire spectrum Align everything to work Sanity / smoke-test type of thing
- New version of orders and exploration
- Ship it! Deploy everything together, all versions are aligned
- Everything is deployed, all's good and aligned
- Fancy books becomes increasingly popular! Sales, marketing says they should expose some of the API's to consumers Dev team: sure, we'll build a simple separate docs app and expose the URLs Ok, it's documented to some extent, marketing can promote it and developers can start using it
- So we have external customers use our API's System to system, external FE clients, business customers, commercial/non commercial use, etc. And then..
- Sales, marketing – how do we know who use it? How can we manage subscriptions, license and bill? How do we know how many use it? Where to invest? Analytics? HA, SLA, Commit? How? Dev – how do we manage changes? (b4 – sync both ends) Stability? Doc it and Communicate? Security (attackers) – DDos, penetration – how can we handle that No standards, hard to adopt Support tickets and calls Hard to move (CLICK) Next: extinguish fires -- --- Now, all kinds of things start to surface How do we license certain API's? How do keep track of KPI's for billing information? How do we monitor API for health? How do we protect the API's and enforce policies? New changes coming in, as usual, change the code and ship it Breaking changes were introduced – not even intentionally nor was it communicated in any way Docs app is outdated SLA? HA? Maintenance procedures? How do we communicate things? How do we version things? Keep things stable? Short-term vs. long-term commit How do we manage the R&D, procedures, pipelines, codebase and the solution as a whole to account for such things? API's use non-consistent nor widely-accepted practices Even within the same API's Hackers try to penetrate our system, are we good? Attackers try to bombard our API's, DDoS, are we good? Support calls coming in at an alarming rate
- Times goes towards extinguishing fires Competitors provide similar services, better controlled, we're starting to lose customers We go bankrupt and our company is terminated
- Times goes towards extinguishing fires Competitors provide similar services, better controlled Customers leaves, We go bankrupt and our company is terminated
- Senior software architect, instructor, speaker and developer Over 20 years of experience in the field Designed and developed various systems, both backend and frontend Mentored and lead large teams and R&D divisions Publications, e.g. Book: HandsOn Full-Stack Web Development with ASP.NET Core Open source: WCF Contrib WCF Microsoft Official Curriculum (MOC) Head of Web and Mobile at CodeValue
- Every bullet can be a lecture by its own, so we’re going to touch only at the high-level
- Over the wire
- Skillset and expertise Technical aspects Technological advantages and constraints Offering, licensing and pricing HTTP/3 – QUIC Needs: What you need to expose Skillset and expertise Technological advantages Interoperability Technical constraints Offering, licensing and pricing
- How much public is considered public? Case study: B2B, inside the same RnD - operational system and automation workflows
- Communication – deprecation notices, release roadmap, maintenance timetable Portals – backoffice and consumer facing In many aspects, a public API is different than an internal API Compatibility and stability Scalability Concurrency Security and validation Rate limiting Etc. You need to meet SLA and contract, even in the long-term
- Provide what is necessary, no more no less Requirement – expose books catalog Use-case – b2b system-to-system sync purposes Catalog hardly changes, blob, API for current month only, slim unaggregated payloads, simple continuation-token paging, etc.
- Routing – facilitate versioning Protocol bridging - HTTP / HTTP2 / gRPC / WebSocket + SSL termination Subscription management – api keys
- APIM – API Mgmt https://dzone.com/articles/api-gateway-vs-service-mesh https://blog.christianposta.com/microservices/do-i-need-an-api-gateway-if-i-have-a-service-mesh/ https://konghq.com/blog/the-difference-between-api-gateways-and-service-mesh
- So things can be more complex. We could have a graph of such components (APIM,Mesh) (ANIM-CLICK) Big R&D, Enterprise divisions, different needs of such components Might lead to multi purpose components and several tools – ingress/horizontal-integration/BFF's Some might be applicative, more towards OSS/DIY, while other could be more infrastructure – use tools/platforms E.g., projects where we build CRD’s + controllers where divisions can spin up API-GW as a managed resource and leverage an existing solution we build for the organization --- Mention Maayan's project There’s Envoy (network proxy) Istio is service mesh which is based on Envoy Proxy There’s Gloo – fork of Envoy – adds enterprise-level features + support + nice convenient product Based on k8s philosophy – give desired state, I’ll bring it there K8s CRD (custom resource definition) – can extend and build your own resource – “API GW” Invent schema for the resource definition, now we can create such resources Need to implement a controller – listens to resource events (created, updated) – responsible to sync between current-desired Maayan’s project took it further, created CRD for certificates, DNS Record, API-GW (gloo edge enterprise) Every division can deploy API-GW as managed k8s resources with the same platform and stack
- Diff integrations – system-to-system / FE Underlying services can have a slice of the complete payload (esp. micro) Consider BFF, BaaS Can be more than just query, application-level logic, might require extensibility/DIY Aggregations – book – author name along with the id https://blog.hubspot.com/website/api-gateway
- Many tools – make sure to evaluate against your needs
- Type of integration B2b, mutual auth - perhaps certs Public, no identity identification – api keys is simple Some SASS/OTS provide key mgmt
- Code first Maintained by developers Implementation is the source of truth
- Contract first Maintained by relevant people Contract (spec) is the source of truth
- Middle ground – spec can be coded e.g., RAML or just code
- Developers implement the application requirements directly There is no contract or specification at first. Requires a strong communication between the teams implementing application parts. If the delivery time matters, a small or an internal API needed to be developed, Code First approach can be the convenient one. Also, a specification or a documentation can be written later for the consumers of the API. Slow feedback loops – Your feedback loop from “start design” to “get feedback” is measured in days or weeks. Even if you adopt a multi-phased implementation approach where you leave rigorous testing until after the initial API design is approved, chances are you’re still going to be implementing a bunch of code before you can get anything up for review. If you need to get feedback from external customers this feedback loop is further extended with deployment cycles etc. Context is spread across multiple places – When you open a pull request, there is no easy way to separate out the API design from the implementation details, and reviewers need to keep the overall API shape in their head as they review multiple source files. Low-value review feedback – Related to #2, review feedback tends to end up being less about the API design and more about implementation details (“I think this should be moved to a separate class,” “Could we introduce some composition here to improve re-use,” etc.). While this is great for the implementation quality, it distracts from the goal of getting a solid API design. Frameworks leak into the design – This is my pet peeve. I often see framework concerns leaking into API design because it makes implementation easier, rather than because it makes the resulting API better. I object to having my frameworks dictate how my API behaves. How many times have you seen comments like “I did it this way because otherwise, I would need a custom Jackson de-serializer“? Your API spec is limited by the available tooling – Many Swagger generators don’t have full spec coverage and it can be painful to coerce them into generating the spec you want. We have had a lot of trouble in Java-land trying to get rid of readOnly flags (why should we break our immutable data model just to make the spec generation happy???) and to be able to specify the maximum number of items in an array (you just can’t do it with the current Swagger annotations).
- Include in review processes Incorporate in DevOps pipelines
- Next: One of the leading versioning practice is.. Semantic versioning
- There’s more – dash (release,hotfix), plus (meta)
- Behavior – input validation – throw on unexpected fields (removing optional is breaking) / SLA – performance changed
- Spec mixed – minor – query string / major – host – both in url Decision factor – http caching Media – Many REST advocates prefer media type versioning as it solves the "one resource, one URI" problem cleanly, and allows adding versioning support after-the-fact. The primary argument against it is the fact that the version is not visible when looking at the URI When we version the Media Type and extend the language, we go through Content Negotiation based on this header. The REST API would make use of custom vendor MIME media types instead of generic media types such as application/json. We're going to version these media types instead of the URIs. What's important to understand here is that the client makes no assumptions about the structure of the response beyond what's defined in the media type. This is why generic media types are not ideal. These do not provide enough semantic information and force the client to use require additional hints to process the actual representation of the resource.
- Let’s talk about different approaches and aspects that can drive how we design our api's
- Innovative – author new ways of doing things Conventional – use industry-founded practices Be conventional if it serves the needs. Look at what others are doing (e.g., advanced query) For example, advanced query Can look up to what other API’s are doing (google, Netflix, etc.) Use widely accepted headers, status codes, etc. REST is not a standard We build RESTful services that adhere to REST principles by using HTTP E.g. of non-RESTful - use-case driven RESTful api’s The more innovative – better docs, more support, etc. Generally – be conventional if it meets your needs
- Rich – let’s expose everything / all functionality. Creates large api surfaces Minimalistic – needs and usecases, commit only to what you need For example – filter/sort – is it really needed? --- Minimalistic – commit only to what you need Meet specific use-cases
- Flexible – all ways of doing things (REST, GQL, gRPC). Paging – skip + top / pageIndex + pageSize / continuation tokens Opinionated - Tunnel clients for specific usage patterns
- Clear – be informative vs. Vague – not to into details E.g., creating a book – email the author. Can be runtime too – validation errors Be clear where it serves a value --- In some areas you may choose to be clear and informative while in others ambiguous If you say something, you commit What do you include in the docs? E.g., idempotence keys This relates to errors and runtime as well Validation errors – do you provide info or not?
- Performance-oriented API’s can affect the API and make it more complicated Or perhaps it’s not really needed and you’ll decide to keep things simple Performance –Large json / string / blob, claim checks, chunky vs. chatty, include/expand, etc. --- Factor API’s performance-oriented only when it serves a real purpose
- Internally, how much freedom do you want to give developers and others? Strict codebase, conventions, workflows -- Permissive - Developers model the API in a way convenient to them
- Stable – on what you decided to commit to Consistent – within your API’s and industry-accepted practices (if it meets your needs) Symmetric – within your API’s, naming, formats, etc. (Book.bookId / User.id / dates) Implementation details – e.g., enums (deleted) and hash functions, etags, continuation tokens (can be opaque)
- Let’s talk about different criteria that can drive our design philosophy
- Opinionated - Tunnel clients for specific usage patterns Informative – docs, error responses Compatibility – expansion over breakage, Try to design in a way which accounts for the roadmap Unstable Commit for version only X months Changes in the app ripple through the API Announce breaking changes regularly Work with 3rd party to adapt frequently Rich Expose everything clients may want Large API surface Unopinionated Let clients act upon their will Just HTTP Developers model the API in a way convenient to them
- We see the APIM lifecycle Every round can be an entire world of tools, technologies and workflows Agility first - Fast delivery, short iterations, minimal specs, quick feedback Diff. lengths --- Gain confidence Like any other product
- Alright, that's a wrap! Next, resources!
- Alright, that's a wrap! Thank you all for listening, I had a blast Enjoy the rest of the conference, and happy coding