Keynote: Sam Newman, Building Microservices | The Tyranny Of Data | Kafka Summit 2020
•
1 gostou•1,510 visualizações
Data is essential. It’s the lifeblood of our business, and without it we’re lost. The problem with data though, is all too often we can end up working for the data, rather than having the data work for us. So much of the systems we build are about managing data properly. Storing it safely, getting it where it needs to be. Making sure it is held safely, or that it is manipulated in the right way. The nature of the data we manage can end up constraining our applications in a host of ways. Rather than making data the constraint in our system, we need to find ways to better unlock the value it has for our organisations. In this keynote, Sam will look at how to reimagine the use of data to make sure the data works for us, not the other way around.
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (16)
Semelhante a Keynote: Sam Newman, Building Microservices | The Tyranny Of Data | Kafka Summit 2020
Semelhante a Keynote: Sam Newman, Building Microservices | The Tyranny Of Data | Kafka Summit 2020 (20)
Mais de HostedbyConfluent
Mais de HostedbyConfluent (20)
Último
Último (20)
Keynote: Sam Newman, Building Microservices | The Tyranny Of Data | Kafka Summit 2020
- 1. THE TYRANNY OF DATA SAM NEWMAN https://www.flickr.com/photos/alexanderkafka/27099276128/
- 6. @samnewman Because it’s important, we’re worried about changing it
- 11. @samnewman
- 12. @samnewman
- 13. @samnewman
- 17. @samnewman So, let’s talk about microservices
- 21. @samnewman Inventory Microservice instances are typically separate processes Shipping V2
- 22. @samnewman Inventory Microservice instances are typically separate processes Communication is via some form of network call Shipping V2
- 23. @samnewman Inventory Microservice instances are typically separate processes Communication is via some form of network call Shipping V2 Data is hidden inside the microservice boundary
- 24. @samnewman “Microservices are an architecture that optimise for autonomy” - James Lewis
- 30. @samnewman INDEPENDENT WORKING Inventory Returns Invoicing Accounts Shipping Team A Customer Service
- 31. @samnewman INDEPENDENT WORKING Inventory Returns Invoicing Accounts Shipping Team A Team B Customer Service
- 32. @samnewman INDEPENDENT WORKING Inventory Returns Invoicing Accounts Shipping Team A Team B Team C Customer Service
- 33. @samnewman INDEPENDENT WORKING Inventory Returns Invoicing Accounts Shipping Team A Team B Team C Customer Service
- 34. @samnewman INDEPENDENT WORKING Inventory Returns Invoicing Accounts Shipping Team A Team B Team C Customer Service V2
- 35. @samnewman INDEPENDENT WORKING Inventory Returns Invoicing Accounts Shipping Team A Team B Team C Customer Service V2
- 36. @samnewman INDEPENDENT WORKING Inventory Returns Invoicing Accounts Shipping Team A Team B Team C Customer Service V2
- 37. @samnewman Maintaining backwards compatibility is key to enabling independent deployability
- 38. @samnewman Let’s look at a cutting edge idea that is key to making modern autonomous organisations work…
- 39. @samnewman …an idea from 1971
- 41. @samnewman COUPLING TO INTERNAL IMPLEMENTATION Accounts
- 42. @samnewman COUPLING TO INTERNAL IMPLEMENTATION Accounts Shipping
- 43. @samnewman COUPLING TO INTERNAL IMPLEMENTATION Accounts If an upstream consumer can reach into your internal implementation.. Shipping
- 44. @samnewman COUPLING TO INTERNAL IMPLEMENTATION Accounts If an upstream consumer can reach into your internal implementation.. Shipping
- 45. @samnewman COUPLING TO INTERNAL IMPLEMENTATION Accounts If an upstream consumer can reach into your internal implementation.. …then you can’t change this implementation without breaking the consumer Shipping
- 46. @samnewman COUPLING TO INTERNAL IMPLEMENTATION Accounts If an upstream consumer can reach into your internal implementation.. …then you can’t change this implementation without breaking the consumer Shipping
- 47. @samnewman COUPLING TO INTERNAL IMPLEMENTATION Accounts If an upstream consumer can reach into your internal implementation.. …then you can’t change this implementation without breaking the consumer Shipping
- 49. @samnewman INFORMATION HIDING! Accounts Shipping Hide your secrets! Hidden
- 50. @samnewman INFORMATION HIDING! Accounts Shipping Hide your secrets! Be explicit about what is shared, and what is hidden Hidden
- 51. @samnewman INFORMATION HIDING! Accounts Shipping Hide your secrets! Be explicit about what is shared, and what is hidden Shared Hidden
- 52. @samnewman INFORMATION HIDING! Accounts Shipping Hide your secrets! Be explicit about what is shared, and what is hidden Shared Hidden Hidden things can change, shared things can’t
- 53. @samnewman INFORMATION HIDING! Accounts Shipping Hide your secrets! Be explicit about what is shared, and what is hidden Shared Hidden Hidden things can change, shared things can’t Shipping
- 54. @samnewman INFORMATION HIDING! Accounts Shipping Hide your secrets! Be explicit about what is shared, and what is hidden Shared Hidden Hidden things can change, shared things can’t Shipping
- 55. @samnewman INFORMATION HIDING! Accounts Shipping Hide your secrets! Be explicit about what is shared, and what is hidden Shared Hidden Hidden things can change, shared things can’t Shipping API
- 56. @samnewman Information hiding makes backwards compatibility so much easier
- 57. @samnewman Hiding data is vital
- 58. @samnewman Splitting apart data allows us to work more in parallel
- 59. @samnewman And gives us more options about how we build our applications
- 60. @samnewman
- 61. @samnewman
- 62. @samnewman
- 63. @samnewman
- 64. @samnewman
- 65. @samnewman
- 66. @samnewman
- 67. @samnewman
- 70. @samnewman Order Updated! 04/08/2020 Customer: Sam Newman Order Value: $130.25 Event
- 71. @samnewman Order Updated! 04/08/2020 Customer: Sam Newman Order Value: $130.25 Event
- 72. @samnewman Order Updated! 04/08/2020 Customer: Sam Newman Order Value: $130.25 Event
- 76. @samnewman
- 81. @samnewman An event stream as an immutable append- only set of records
- 83. @samnewman But isn’t sharing events like sharing a database?
- 84. @samnewman Events are things you have consciously decided to share
- 85. @samnewman But what about joins? Ad-hoc queries?
- 86. @samnewman
- 87. @samnewman
- 89. @samnewman REPORTING DATABASE PATTERN https://www.martinfowler.com/bliki/ReportingDatabase.html Publish data to a dedicated database for reporting
- 90. @samnewman REPORTING DATABASE PATTERN https://www.martinfowler.com/bliki/ReportingDatabase.html Publish data to a dedicated database for reporting Allows for joins, ad-hoc queries in a familiar setting
- 91. @samnewman REPORTING DATABASE PATTERN https://www.martinfowler.com/bliki/ReportingDatabase.html Publish data to a dedicated database for reporting Allows for joins, ad-hoc queries in a familiar setting And Kafka can help here!
- 92. @samnewman
- 96. @samnewman What about legacy applications?
- 97. @samnewman PATTERN: CHANGE DATA CAPTURE Existing Monolith Commit
- 98. @samnewman PATTERN: CHANGE DATA CAPTURE Existing Monolith Commit
- 99. @samnewman PATTERN: CHANGE DATA CAPTURE Existing Monolith Commit CDC Tool
- 100. @samnewman PATTERN: CHANGE DATA CAPTURE Existing Monolith Commit CDC Tool Capture data as its inserted
- 101. @samnewman PATTERN: CHANGE DATA CAPTURE Existing Monolith Commit CDC Tool Capture data as its inserted Different tools can support different destinations
- 102. @samnewman PATTERN: CHANGE DATA CAPTURE Existing Monolith Commit CDC Tool Capture data as its inserted Different tools can support different destinations
- 103. @samnewman DEBEZIUM, FOR FUN AND (HOPEFULLY) PROFIT https://debezium.io/
- 105. @samnewman Can’t we do better than this?
- 109. @samnewman PUSH QUERY Order Order Placed
- 110. @samnewman PUSH QUERY Order Order Placed Total Orders Placed Today:
- 111. @samnewman PUSH QUERY Order Order Placed $1509Total Orders Placed Today:
- 112. @samnewman PUSH QUERY Order Order Placed $1670Total Orders Placed Today:
- 113. @samnewman PUSH QUERY Order Order Placed $2440Total Orders Placed Today:
- 114. @samnewman PUSH QUERY Order Order Placed $2789Total Orders Placed Today:
- 115. @samnewman
- 116. @samnewman PUSH QUERY - CROSS STREAM Order Order Placed
- 117. @samnewman PUSH QUERY - CROSS STREAM Order Order Placed Catalog Item Added
- 118. @samnewman PUSH QUERY - CROSS STREAM Order Order Placed Catalog Item Added
- 119. @samnewman PUSH QUERY - CROSS STREAM Order Order Placed Catalog Item Added Genre Sales Today Sales This Week
- 120. @samnewman PUSH QUERY - CROSS STREAM Order Order Placed Catalog Item Added Genre Sales Today Sales This Week Rock A$1209 A$6815
- 121. @samnewman PUSH QUERY - CROSS STREAM Order Order Placed Catalog Item Added Genre Sales Today Sales This Week Rock A$1209 A$6815 Hip Hop A$1570 A$10837
- 122. @samnewman PUSH QUERY - CROSS STREAM Order Order Placed Catalog Item Added Genre Sales Today Sales This Week Rock A$1209 A$6815 Hip Hop A$1570 A$10837 Death Polka A$2570 A$12725
- 123. @samnewman PULL QUERIES Order Order Placed
- 124. @samnewman PULL QUERIES Order Order Placed What’s the most recent order by Sam?
- 125. @samnewman Kafka and KSQL - making sure your data works for you
- 128. @samnewman THANKS!