1.
Sai Srirampur | PyConCA 2018Sai Srirampur | PyConCA 2018
Scaling Multi-Tenant
Applications Using the
Django ORM & Postgres
Sai Srirampur
PyCon Canada | Toronto | Nov 2018

2.
Sai Srirampur | PyConCA 2018
• Sai Srirampur a.k.a Sai
• Engineer at Citus Data
• Joined Citus to make it so
developers never have to
worry about scaling their
database
• Creator django multi-tenant
• Follow me @saisrirampur
@citusdata

3.
Sai Srirampur | PyConCA 2018

4.
Sai Srirampur | PyConCA 2018
Talk about my favorite things
Sai Srirampur | PyConCA 2018

5.
Sai Srirampur | PyConCA 2018Sai Srirampur | PyConCA 2018
PYTHON &
DJANGO
POSTGRES SCALING
MULTI-TENANT
APPLICATIONS

6.
Sai Srirampur | PyConCA 2018Sai Srirampur | PyConCA 2018
Why 💙 Postgres? TLDR;
Open source
Constraints
Extensions
PostGIS / Geospatial
HLL, TopN, Citus
Foreign data wrappers
Rich SQL
CTEs
Window functions
Full text search
Datatypes
JSONB

7.
Sai Srirampur | PyConCA 2018
Today… Scaling Multi-Tenant Apps
Using the Django ORM & Postgres
3 architectures to build Multi-tenant apps
django_multitenant
Scaling multi-tenant apps with distributed
postgres

8.
Sai Srirampur | PyConCA 2018
• Multiple customers
(“tenants”)
• Each with own data
• SaaS
• Shopify, Salesforce
Multi-Tenant Apps
Sai Srirampur | PyConCA 2018

9.
Sai Srirampur | PyConCA 2018Sai Srirampur | PyConCA 2018
Why talk
about scalable
multi-tenant
applications?
tens
100’s
1000’s

10.
Sai Srirampur | PyConCA 2018Sai Srirampur | PyConCA 2018
Architectures
To
Build
Multi-Tenant
Applications3

11.
Sai Srirampur | PyConCA 2018
[1] One database per
tenant
Sai Srirampur | PyConCA 2018

12.
Sai Srirampur | PyConCA 2018
What defines a Database?
• Organized collection of interrelated data
• Don’t share resources.
• Username and password
• Connections
• Memory

13.
Sai Srirampur | PyConCA 2018
One database per tenant
Tenant 1251 Tenant 1252
Tenant 5
[1]

14.
Sai Srirampur | PyConCA 2018Sai Srirampur | PyConCA 2018
CREATE DATABASE tenant_100;
./manage.py migrate --database=tenant_100;
One database per tenant / Onboarding
database_routing: db_for_read, db_for_write etc.

15.
Sai Srirampur | PyConCA 2018
● Start quickly
● Isolate customer (tenant) data
● Compliance is a bit easier
● Time for DBA/developer to manage
● Maintain consistency (ex: create
index across all databases)
● Longer running migrations
● Performance degrades as #
customers (tenants) goes up
PROS CONS
[1] One Database Per Tenant
Sai Srirampur | PyConCA 2018

16.
Sai Srirampur | PyConCA 2018
[2] One schema per
tenant
Sai Srirampur | PyConCA 2018

17.
Sai Srirampur | PyConCA 2018
What defines a database Schema?
• Logical namespaces to hold a set of tables
• Share resources:
• Username and password
• Connections
• Memory

18.
Sai Srirampur | PyConCA 2018
One schema per tenant
Tenant 5 Tenant 1251
Tenant 1252
Database
[2]

19.
Sai Srirampur | PyConCA 2018Sai Srirampur | PyConCA 2018
CREATE schema tenant_100;
./manage.py migrate --schema=tenant_100;
app_code for schema routing based on tenant
One schema per tenant / Onboarding

20.
Sai Srirampur | PyConCA 2018
● Better resource utilization vs. one
database per tenant
● Start quickly
● Logical isolation
● Hard to manage (ex: add column
across all schemas)
● Longer running migrations
● Performance degrades as #
customers (tenants) goes up
PROS CONS
[2] One Schema Per Tenant
Sai Srirampur | PyConCA 2018

21.
Sai Srirampur | PyConCA 2018
[3] Shared table
architecture
Sai Srirampur | PyConCA 2018

22.
Sai Srirampur | PyConCA 2018
Accounts
Shared table architecture
Campaigns
Leads TenantId
5
5
5
1251
Database
[3]

23.
Sai Srirampur | PyConCA 2018
Example

24.
Sai Srirampur | PyConCA 2018
● Easy maintenance
● Faster running migrations
● Best resource utilization
● Faster performance
● Scales to 1k-100k tenants
● Application code to guarantee
isolation
● Make sure ORM calls are always
scoped to a single tenant
PROS CONS
[3] Shared Table Architecture :-)
Sai Srirampur | PyConCA 2018

25.
Sai Srirampur | PyConCA 2018
Comparing the 3 architectures for scaling
multi-tenant Django applications
ONE
DATABASE
PER TENANT
ONE
SCHEMA
PER TENANT
SHARED
TABLE
ARCHITECTURE
Database
Database
[1]
[2]
[3]
start quickly & isolation guarantees,
bad resource utilization & not scalable
better resource util, logical isolation
not scalable
Scales to over 100K tenants!
explicitly handle isolation

26.
Sai Srirampur | PyConCA 2018

27.
Sai Srirampur | PyConCA 2018
django_multitenant to the rescue
Sai Srirampur | PyConCA 2018

28.
Sai Srirampur | PyConCA 2018Sai Srirampur | PyConCA 2018
django_multitenant
Automates all ORM calls
to be scoped to a single tenant

29.
Sai Srirampur | PyConCA 2018
Today...
Purchase.objects.filter(id=1)
<=>
"SELECT* from purchase where id=1"

30.
Sai Srirampur | PyConCA 2018Sai Srirampur | PyConCA 2018
With django_multitenant
Purchase.objects.filter(id=1)
<=>
"SELECT* from purchase where id=1 and
store_id=<current_tenant>"

31.
Sai Srirampur | PyConCA 2018
Components of
django_multitenant
Sai Srirampur | PyConCA 2018

32.
Sai Srirampur | PyConCA 2018
django_multitenant usage — 3 steps
1. Inherit all models with TenantModel
2. Change ForeignKey to TenantForeignKey
3. Define tenant scoping: set_current_tenant(t)

33.
Sai Srirampur | PyConCA 2018
TenantModel defines new Manager
Sai Srirampur | PyConCA 2018

34.
Sai Srirampur | PyConCA 2018
TenantForeignKey mimics composite
foreign key behavior
• Adds tenant_id filter to referenced model. Handles:
• Reference lookups — ex: (Purchase.product.name)
• select_related() / prefetch_related()
• Explicit Joins (product__name)

35.
Sai Srirampur | PyConCA 2018
set_current_tenant(t)
• Specifies which tenant the APIs should be scoped to
• Set at authentication logic via middleware
• Set explicitly at top of function (ex. view, external
tasks/jobs)

36.
Sai Srirampur | PyConCA 2018
Example Code
Sai Srirampur | PyConCA 2018

37.
Sai Srirampur | PyConCA 2018
Models without django_multitenant
class Purchase(models.Model):
store = models.ForeignKey(Store)
product_purchased = models.ForeignKey(Product)
ordered_at = models.DateTimeField(default=timezone.now)
billing_address = models.TextField()
Sai Srirampur | PyConCA 2018

38.
Sai Srirampur | PyConCA 2018
Post django_multitenant
class Purchase(TenantModel):
store = models.ForeignKey(Store)
product_purchased = TenantForeignKey(Product)
ordered_at = models.DateTimeField(default=timezone.now)
billing_address = models.TextField()
Sai Srirampur | PyConCA 2018

39.
Sai Srirampur | PyConCA 2018
Setting the tenant at authentication
class SetCurrentTenantFromUser(object):
def process_request(self, request):
if not hasattr(self, 'authenticator'):
from rest_framework_jwt.authentication import
JSONWebTokenAuthentication
self.authenticator = JSONWebTokenAuthentication()
try:
user, _ = self.authenticator.authenticate(request)
except:
return
try:
#Assuming your app has a function to get the tenant associated
for a user
current_tenant = get_tenant_for_user(user)
except:
# TODO: handle failure
return
set_current_tenant(current_tenant)
Sai Srirampur | PyConCA 2018

40.
Sai Srirampur | PyConCA 2018

41.
Sai Srirampur | PyConCA 2018
Benefits of django_multitenant
Drop-in implementation of shared tables architecture
Guarantees isolation
Ready to scale with distributed Postgres (Citus)

42.
Sai Srirampur | PyConCA 2018

43.
Sai Srirampur | PyConCA 2018
Infinite scale
with Citus

44.
Sai Srirampur | PyConCA 2018
django_multitenant with Postgres

45.
Sai Srirampur | PyConCA 2018
django_multitenant with Citus

46.
Sai Srirampur | PyConCA 2018Sai Srirampur | PyConCA 2018
django_multitenant with Citus

47.
Sai Srirampur | PyConCA 2018
also one of my favorites: Citus

48.
Scale out Django!
github.com/citusdata/django-multitenant
@saisrirampur @citusdata
sai@citusdata.com
citusdata.com/newsletter
Thank You