O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
Implementing
Gannett's next
generation CMS
Using CQRS with Event Sourcing
Supported by Couchbase
Who we are
• We exist to get the right information, tools and guidance to people at
the right time.
• We serve communities...
Who we are
• 108 news rooms across the country
• ~3000 contributors
• > 100 million unique visitors/month
Original Architecture
How we query for a site layout
• 100+ stored procs
• Some 400+ lines
• Business logic
• Not incld server side
Challenges
• Slow response times for all consumers
• Slow time to publish content – critical in our domain
• Heavy caching...
Goals
• Migrate a legacy system to the cloud
• Decrease publishing latency
• Increase performance, reliability
• Decrease ...
Guiding Principles
• Not everything should be crammed into a relational model
• Write it as you want to read it
• Separate...
Why Couchbase?
• Read performance, optimize for more reads than writes
• Fault Tolerance
• N1QL
• NodeJS integration
• Eas...
A word about performance
• Minimum required ~500 writes and 3000 reads/second
• Successful Load Test
– 10,000 writes/secon...
Solution
• Messaging pipeline (Integration)
• Presentation environment (Query)
• Authoring environment (Command)
Messaging Pipeline
Presentation System
Authoring System
No software survives
contact with reality
• Problems with N1QL eventual consistency
• Turns out a lot happens to our entit...
Authoring System Phase 2
Put it all
together
Presentation Couchbase
• 5 nodes
• r3.2xlarge
• 8 vCPUs
• 61GB RAM
• 160GB SSD
• 234GB total cache size
• 2.4TB total stor...
Presentation Couchbase Buckets
• Other Buckets
– 2k gets/sec
– 50 sets/sec
– 500,000 items
– cache miss ratio: ~0
– active...
Presentation API Metrics
• 90th percentile response times
– Site layout gets: ~170ms
– Search (various APIs): 90–350ms
– A...
Authoring Couchbase
• 8 nodes
• r3.2xlarge
• 8 vCPUs
• 61GB RAM
• 160GB SSD
• 400GB total cache size
• 23.4TB total storag...
Authoring Couchbase Buckets
• Snapshot Store
– 2k gets/sec
– 330 sets/sec
– 150 million items
– Cache miss ratio: <1
– act...
Authoring API Metrics
• 90th percentile response times
– Commands: 176ms
– Gets: 319ms
– Search: 771ms
• Median
– Commands...
Lessons Learned
• Strong, effective leadership is required
• For new, large projects: have a kickoff!
• Long-term roadmaps...
Lessons Learned
• TESTING!!!
– Unit test
– Integration test
• Module integration
• Application integration
• System integr...
Next Steps
• Complete the rollout to all markets
• Complete feature development
• Decommission legacy system
• Further tun...
Next Steps Continued
• Next event store implementation
• Multi-region Robust HA
• End to end transaction tracking
• Domain...
To be continued…
Próximos SlideShares
Carregando em…5
×

Implementing Gannett's next-generation CMS with CQRS and event sourcing using Couchbase – Connect Silicon Valley 2017

777 visualizações

Publicada em

Speaker: Justin Haugh, Principal Developer, Gannet Co., Inc.

When the limits of Gannett's existing content management platform started to become evident, a new solution was sought to better execute the mission of our content creators and editors. We needed to design a system that provided extremely fast responses to our front-end consumers, as well as a more responsive and informative content management system for our newsrooms. We elected to use Couchbase as our data storage solution. This is our journey.

Publicada em: Tecnologia
  • If we are speaking about saving time and money this site ⇒ www.HelpWriting.net ⇐ is going to be the best option!! I personally used lots of times and remain highly satisfied.
       Responder 
    Tem certeza que deseja  Sim  Não
    Insira sua mensagem aqui
  • My friend sent me a link to to tis site. This awesome company. They wrote my entire research paper for me, and it turned out brilliantly. I highly recommend this service to anyone in my shoes. ⇒ www.HelpWriting.net ⇐.
       Responder 
    Tem certeza que deseja  Sim  Não
    Insira sua mensagem aqui
  • I can advise you this service - ⇒ www.WritePaper.info ⇐ Bought essay here. No problem.
       Responder 
    Tem certeza que deseja  Sim  Não
    Insira sua mensagem aqui
  • Seja a primeira pessoa a gostar disto

Implementing Gannett's next-generation CMS with CQRS and event sourcing using Couchbase – Connect Silicon Valley 2017

  1. 1. Implementing Gannett's next generation CMS Using CQRS with Event Sourcing Supported by Couchbase
  2. 2. Who we are • We exist to get the right information, tools and guidance to people at the right time. • We serve communities of all kinds and the diverse individuals and businesses that form them. • Our media drives action, not passive consumption. It helps people and businesses make meaningful change.
  3. 3. Who we are • 108 news rooms across the country • ~3000 contributors • > 100 million unique visitors/month
  4. 4. Original Architecture
  5. 5. How we query for a site layout • 100+ stored procs • Some 400+ lines • Business logic • Not incld server side
  6. 6. Challenges • Slow response times for all consumers • Slow time to publish content – critical in our domain • Heavy caching for front end consumption (~16 minutes) • On-prem pet servers – difficulty scaling​ – protected by DBAs and Release Managers • Replication issues with SQL Server​ • Business logic everywhere • Lack of feedback from transactions
  7. 7. Goals • Migrate a legacy system to the cloud • Decrease publishing latency • Increase performance, reliability • Decrease cost • Consolidate and organize business logic
  8. 8. Guiding Principles • Not everything should be crammed into a relational model • Write it as you want to read it • Separate reads and writes to scale independently (CQRS) • Domain Driven Design for modeling behaviors • Event Driven Design for implementing behaviors
  9. 9. Why Couchbase? • Read performance, optimize for more reads than writes • Fault Tolerance • N1QL • NodeJS integration • Ease of scaling
  10. 10. A word about performance • Minimum required ~500 writes and 3000 reads/second • Successful Load Test – 10,000 writes/second – 30,000 reads/second – spikes of up to 330,000 reads​/second – 8 r3.xlarge nodes (Amazon) – 32 vCPU – 172 GB Memory Allocated
  11. 11. Solution • Messaging pipeline (Integration) • Presentation environment (Query) • Authoring environment (Command)
  12. 12. Messaging Pipeline
  13. 13. Presentation System
  14. 14. Authoring System
  15. 15. No software survives contact with reality • Problems with N1QL eventual consistency • Turns out a lot happens to our entities
  16. 16. Authoring System Phase 2
  17. 17. Put it all together
  18. 18. Presentation Couchbase • 5 nodes • r3.2xlarge • 8 vCPUs • 61GB RAM • 160GB SSD • 234GB total cache size • 2.4TB total storage size
  19. 19. Presentation Couchbase Buckets • Other Buckets – 2k gets/sec – 50 sets/sec – 500,000 items – cache miss ratio: ~0 – active resident: 100% – value ejection • Assets – 100k get/sec sustained – >150k get/sec burst – 30 set/sec sustained – 300 set/sec burst – 200 million items – cache miss ratio: < 0.5 – active resident: 9% – value ejection
  20. 20. Presentation API Metrics • 90th percentile response times – Site layout gets: ~170ms – Search (various APIs): 90–350ms – Analytics based APIs: 400-800ms • Median – Site layout gets: ~55-65ms – Search (various APIs): ~15-50ms – Analytics based APIs: ~300-600ms • Published content is surfaced within seconds
  21. 21. Authoring Couchbase • 8 nodes • r3.2xlarge • 8 vCPUs • 61GB RAM • 160GB SSD • 400GB total cache size • 23.4TB total storage size
  22. 22. Authoring Couchbase Buckets • Snapshot Store – 2k gets/sec – 330 sets/sec – 150 million items – Cache miss ratio: <1 – active resident: 3% – value ejection • Event Store – 16k gets/sec – 70 sets/sec – 350 million items – cache miss ratio: ~2 – active resident: 6% – full ejection
  23. 23. Authoring API Metrics • 90th percentile response times – Commands: 176ms – Gets: 319ms – Search: 771ms • Median – Commands: 26ms – Gets: 64ms – Search: 390ms
  24. 24. Lessons Learned • Strong, effective leadership is required • For new, large projects: have a kickoff! • Long-term roadmaps • Silo teams are poison • CQRS and DDD – live and breath the domain
  25. 25. Lessons Learned • TESTING!!! – Unit test – Integration test • Module integration • Application integration • System integration – Load test – Stress test – UI module tests
  26. 26. Next Steps • Complete the rollout to all markets • Complete feature development • Decommission legacy system • Further tuning on Authoring Couchbase
  27. 27. Next Steps Continued • Next event store implementation • Multi-region Robust HA • End to end transaction tracking • Domain Driven Design is challenging • Containerization of stateless applications
  28. 28. To be continued…

×