How to implement a truly modular ecommerce platform on the example of Spryker at code.talks commerce special 2017

How to implement a truly modular
ecommerce pla2orm
code.talks commerce special 2017

2017 Spryker Systems GmbH
Mo>va>on

About me
Fabian Wesner
CTO @ Spryker Systems

TwiGer: @FabianWesner

hGp://start.spryker.com

What is Spryker?
SPRYKER is a Commerce Opera>ng System
for customer or machine ini>ated transac>ons.

Modularity
One of Spryker’s key characteris>cs:

Modularity

This talk is about learnings, tradeoﬀs, pa5erns, tools and
principles to build a truly modular ecommerce pla9orm.

Monolith or Modules or
Microservices?

Monolith vs Modules vs Microservices

Microservices are compelling
•  Independent teams, hos>ng and deployments
•  Reduced complexity for single services
•  Free choice of technologies….
•  Big challenge. New tools to discover.
•  Great for conference talks

But: Distributed compu>ng is hard
•  Asynchronous calls
•  Network is not reliable and has latency
•  Debuging via network
•  Performance issues
•  API versioning
•  Authen>ﬁca>on
•  Service discovery
•  Distributed logging
•  Eventual consistency, no simple transac>ons
•  Distributed sets of data, no straight-‐forward joins
•  No protec>on by foreign key constraints
•  …

Good use case for Microservices
Microservices can work very well when there are very clean boundaries and communica>on direc>ons.
Example: Pla,orm for Online Games
Free choice of technology per game.
Independent deployment, hos>ng, data-‐
storage, etc.

Basic func>onality is implemented in a shared
service (e.g. Customer, Payment, …)

In this scenario Microservice is natural choice,
even for companies that are not the scale of
Google or Amazon.

Bad use case for Microservices
Microservices become pain in the ass when the system has high amount of internal dependencies
which usually becomes visible in the ER diagram of the database schema.
Example: Online-‐Shop
The diagram shows a typical
database schema of a online shop.

All rela>onships are meaningful and
need to be protected by the
database.

It’s not even big!

Modularity

Prinicples of modularity
Strong encapsula/on
Hide implementa>on details inside components, leading to low coupling between different parts.
Teams can work in isola>on on decoupled parts of the system.

Well-‐defined interfaces
You can't hide everything (or else your system won't do anything meaningful), so well-‐defined and
stable APIs between components are a must. A component can be replaced by any implementa>on
that conforms to the interface specifica>on.

Explicit dependencies
Having a modular system means dis>nct components must work together. You'd beGer have a good
way of expressing (and verifying) their rela>onships.
Quoted from hGps://www.oreilly.com/ideas/modules-‐vs-‐microservices

Bad news!

PHP does not have built-‐in support for modules!

So we need to solve it with conven>ons, paGerns and the help of composer.

Programming modules

What is a bundle (~module)?
Spryker’s defini/on of a bundle

•  “A bundle is a func>onal unit!”
•  It defines its internal API
•  It defines its data structures
•  It defines its dependencies
•  It contains all required layers:
•  Presenta>on
•  Communica>on
•  Business
•  Persistence
•  It contains all unit tests

Bundles are small (~ micro),
coherent and loose coupled.

There will be a high numer
of bundles! (> 100)

We need a consistent way
how to communicate
between bundles.
è

Facades as internal API for bundles
Client
Facade!
Cart!
Facade pa5ern is structural pa5ern by the GOF.

Facade interface

Bundle to Bundle Communica>on
Facade! Facade!
Cart! Calculation!
Facade!
Discount!
calculateDiscounts()!recalculate()!add()!

Facade interface challenge
Internal class CartOpera>on programs against interface of Calcula>onFacade
Where are the bundle boundaries?

Approach 1: cart-‐opera>on class and interface are in same bundle.
Problem: The facade needs to implement an interface from another bundle.

Approach 2: Facade and interface are in the same bundle
Problem: The Opera>on class has a hard-‐coded dependency to a speciﬁc
bundle.

Solu>on with bridge paGern
Solu/on: Add a bridge-‐paGern and a second interface to solve the problem!
The CartToCalcula>on contains only the required methods (ISP).

The bridge allows us to
use any class that ﬁts to
the interface.

Coupling in run-‐>me only.
Not in development-‐>me.

Facade of Calcula>on-‐bunde is located and becomes wrapped into the
CartToCalcula>on-‐bridge.

Shared data structure
Challenge: Data needs to be exchanged between bundles.
Approach: Usage of arrays!?
public function recalculate(array $quote)
{
return $this->getFactory()->createStackExecutor()->recalculate($quote);
}
Not expressive.
No clear structure.

Challenge: Data needs to be exchanged between bundles.
Approach: Data transfer objects (DTOs)

Challenge: Where to put these DTOs?
Approaches:
•  Have the same DTO in both bundle
•  Have the DTO in the “main” bundle (Which one is it?)

Solu/on: Deﬁne the DTO in XML in all related bundles, merge them and
generate the object.
è

Summary Communica>on
Bundles talk to each other via
Facades (~ internal APIs)

Bridges and interfaces connect
the bundles in run-‐>me but keep
them decoupled in development-‐>me.

Data is transported via Data transfer objects

Modular database
In a perfect world every bundle / service has it’s own database and data is
only transmiGed via messages.
But…

Modular database
In an e-‐commerce applica>on things
are connected.

Real-‐world use cases
•  Show all customers with their number of sales-‐orders
•  Discounts are valid only for products with a high stock
•  Discounts are valid only for speciﬁc customers
•  Products have speciﬁc prices per customers (Typical B2B feature)
•  …
What we need is normalized database with protected foreign-‐key rela>ons!

Modular database
Challenge: How to handle cross-‐bundle rela>ons?

Example: “A product has stock”
•  Table spy_product belongs to Product-‐bundle
•  Tables spy_stock_product and spy_stock belong to Stock-‐bundle

Modular database
Solu/on: Each bundle ships with it’s own schema defini>on!
All XML-‐files are merged, diffed and migrated to the database.
When a bundle is removed , all the related tables and rela>ons are gone as well.

Modular database
Solu/on: Each bundle ships with it’s own schema deﬁni>on!

Discussion:

•  Every bundle can query it’s own data only.
•  Cross-‐bundle joins are possible but they create a >ght coupling.
•  Therefore we prefer communica>on via messages.
•  The main reason to do cross-‐bundle joins is performance. It’s a tradeoﬀ.

Designing Dependencies

Designing dependencies
•  Product bundle holds the product en>>es
•  Stock bundle manages the inventory

What are their dependencies?

Designing dependencies
Considera>ons:
•  There can be products without stock
•  There is never a stock without a product

è  Stock-‐bundle requires Product-‐bundle
(~ Product-‐bundle is responsible for Stock-‐bundle)

Another simple example (availability)
Availability of a product = Stock – Sold Items
Bundle Responsibility
Availability-‐bundle Calculates the availability of a product
Oms-‐bundle Order Management System knows the state of an order-‐item
Product-‐bundle

Holds the product en>>es

Stock-‐bundle

Holds the stock of a product

We need several bundles:

What are their dependencies?

Some>mes it‘s not that simple
Use case: Discounts are valid for speciﬁc customer-‐groups

What are the dependencies?

What are the dependencies?
•  Both bundles can be used separately!
•  Discounts are possible without any customer-‐groups and vice-‐versa.

Run-‐/me call: Discount-‐bundle needs to check if current customer belongs to
the related Customer-‐Group
Does that mean that Discount-‐bundle requires Customer-‐Group? No!

We put a Connector in the middle!
Now both bundles are independent from each other BUT can s>ll work
together.

How does that technically work?

✔ The connector makes a direct call to the CustomerGroupFacade

? But how can we make a call from discount-‐bundle to the connector with a
direct dependy?

Inversion of control
Solu=on: Inversion of control with Plugins

Plugin conﬁgura>on
Solu=on: Inversion of control with Plugins
The plugin is conﬁgured into the Discount-‐bundle!

Learning: Dependencies != Coupling
Conceptual dependencies are given and need to be discovered by the team
•  Stock requires Product
•  Product-‐OpTon requires Product
•  Customer-‐Group requires Customer
•  Etc.

Technical coupling between bundles doesn’t just happen!
•  Dependencies can be inverted if needed.
•  Dependencies can be mandatory or op>onal

Experience
Although we managed our dependencies very carefully, we ended up with
this:

Learning
We need to obey the Principles of Package Design!

Principles of cohesion
•  The Release/reuse equivalence principle
•  The Common reuse principle
•  The Common closure principle
Principles of coupling
•  The Acyclic dependencies principle
•  The Stable dependencies principle
•  The Stable abstrac>ons principle

Recommended books
The classic book about agile
sooware design from Robert Mar>n.
A modernized explana>on of SOLID
and Package Principles for modern PHP
development by MaGhias Noback.

The Acyclic dependencies principle
The dependency graph of packages must have no cycles.

The Acyclic dependencies principle

The Stable dependencies principle
Depend in the direc>on of stability.

What is Stability?
A bundle is more stable the more bundles requires it.
Stability = Outgoing Dependencies / (Outgoing + Incoming Dependencies)

Stability measurement in Spryker

Aoer cleanup
Dependency Cleanup was possible by splipng bundles and inver>ng
dependencies.

Real world dependencies (Spryker)

Beneﬁts
A clean dependency tree has a lot of advantages:

•  Reduced system complexity. Avoidance of side-‐eﬀects.
•  Team can work independently even with large systems.
•  It becomes possible to split the applica>on into services
•  Single bundles can be released, replaced and extended

Managing Dependencies

Bundle organiza>on
Every bundle has it’s own:
•  Repository

•  Seman>c version number
•  Composer.json

Example: Spryker Cart

Composer.json
We use Composer.json in the following way:

Require For direct dependencies (~ Calls to a facade)

Suggest For plugins that oﬀer addi>onal func>onality

Require-‐Dev For dependencies that only exist in tests.

Challenge: Slow composer update
Spryker releases in >120
bundles now!

And many more in the
future!

Problem: Composer
update takes ages….

Solu>on: Toran Proxy
We run a public Toran Proxy that pre-‐fetches the composer.json
ﬁles from Github and allows to execute composer update in < 1
minute.
In case you prefer SaaS, then public or private Packagist will do the same job.

Challenge: Core development
How does core development work with > 120 bundles?

Should we commit to all bundles?

What about branches and tags?

What about conﬂicts?

How do release features?

Solu>on: Git Subtree Split
Git has a handy tool for this problem. Git Subtree Split:

•  The core team works with a single (private repository).

•  There is a liGle app installed on a server that reacts on every
commit to master and executes the subtree split opera>on.

•  The opera>on maps single directories to other repositories and
forwards the commits.

Solu>on: Git Subtree Split
You can start with the open source bash script from Fabien Potencier:
hGps://github.com/splitsh/lite

How to release > 120 bundles?
There are two approaches:

Tags in the source-‐repository are forwarded to all splits
•  Every bundle has all versions
•  Some versions contain no changes for a single bundle (Phantom release)
•  Advantage: There is a global version number (e.g. Symfony 3.0.0)
•  This is how Symfony does it
Tags in the source-‐repository are forwarded only to these splits that contain a change
•  Bundles have diﬀerent versions
•  Advantage: Bundles can be released independently (Atomic Release)
•  This is how Spryker does it

STRICTLY CONFIDENTIAL
74
You cannot solve 21st century e-‐commerce problems with 20th century technology
www.spryker.com
@sprysys

How to implement a truly modular ecommerce platform on the example of Spryker at code.talks commerce special 2017

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to How to implement a truly modular ecommerce platform on the example of Spryker at code.talks commerce special 2017

Similar to How to implement a truly modular ecommerce platform on the example of Spryker at code.talks commerce special 2017 (20)

Recently uploaded

Recently uploaded (20)

How to implement a truly modular ecommerce platform on the example of Spryker at code.talks commerce special 2017