This talk explains how to perform SQL query analysis and how to rewrite your views to reduce the number of queries Django uses in evaluating your model objects and their attributes. Special emphasis will be given to the powerful methods "select_related" and "prefetch_related." I will highlight the problem with a naive use of the ORM, how to target code for optimization, and the beneficial result. Like any abstraction layer, the Django ORM hides the messy details about its underlying implementation. This is both the benefit and the risk. If used naively, any tool can cause unexpected or problematic outcomes. Likewise, the ORM can cause your application to interact with the database in an ugly and inefficient way, creating special challenges regarding scaling a quickly-prototyped webapp. Many design patterns and best practices have been developed as a result to nudge developers to use the ORM more efficiently. The good news is, one of the easiest and most powerful patterns has been wrapped into Django itself, in the dual pairs of methods in the Django ORM's Queryset API, "select_related" and "prefetch_related." These methods instruct the Queryset, when evaluated, to perform two kinds of useful optimizations for you that can reduce the number of queries by orders of magnitude resulting from iterating over model objects and many-to-many relations. This talk summarizes the problem these methods of the Queryset API try to solve, how to effectively use them, and the beneficial result. Mastering how to use Queryset methods efficiently and powerfully is a major step in moving from a beginner to intermediate Django developer. Delivered at DjangoCon 2014.

1. A Related Matter:
Optimizing your webapp by using django-debug-toolbar,
select_related(), and prefetch_related()
Christopher Adams
DjangoCon 2014
https://github.com/adamsc64/a-­‐related-­‐matter

2. Christopher Adams
• Software Engineer at Venmo
• Twitter/Github: @adamsc64
• I’m not Chris Adams (@acdha), who works at
Library of Congress
• Neither of us are “The Gentleman” Chris Adams
(90’s-era Professional Wrestler)

4. Django is great
But Django is really a set of tools

5. Tools are great
But tools can be used in good or bad ways

6. The Django ORM:
A set of tools

7. Manage your own
expectations for tools
• Many people approach a new tool with broad set
of expectations as to what the think it will do for
them.
• This may have little correlation with what the
project actually has implemented.

8. As amazing as it
would be if they did…

9. Unicorns don’t exist

10. The Django ORM:
An abstraction layer

11. Abstraction layers
• Great because they take us away from the
messy details
• Risky because they take us away from the
messy details

12. Don’t forget
You’re far from the ground

13. The QuerySet API

14. QuerySets are Lazy

15. QuerySets are
Immutable

16. Lazy: Does not evaluate
until it needs to

17. Immutable: Never
itself changes

18. Each a new QuerySet, none
hit the database
• queryset
=
Model.objects.all()
• queryset
=
queryset.filter(...)
• queryset
=
queryset.values(...)

19. Hits the database
(QuerySet is “evaluated”):
• queryset
=
list(queryset)
• queryset
=
queryset[:]
• for
model_object
in
queryset:
do_something(...)

20. Our app: a blog

21. Models
class Blog(models.Model):!
submitter = models.ForeignKey('auth.User')!
!
class Post(models.Model):!
blog = models.ForeignKey('blog.Blog', related_name="posts")!
likers = models.ManyToManyField('auth.User')!
!
class PostComment(models.Model):!
submitter = models.ForeignKey('auth.User')!
post = models.ForeignKey('blog.Post', related_name="comments")!

22. List View
def blog_list(request):!
blogs = Blog.objects.all()!
return render(request, "blog/blog_list.html", {!
"blogs": blogs,!
})!

23. List Template

25. Detail View
def blog_detail(request, blog_id):!
blog = get_object_or_404(Blog, id=blog_id)!
posts = Post.objects.filter(blog=blog)!
return render(request, "blog/blog_detail.html", {!
"blog": blog,!
"posts": posts,!
})!

26. Detail Template

28. SQL Queries?

29. If you can’t measure
it…

30. …you’d never know if
there are problems.

32. First view:
The blog list page

38. select_related()
• select_related works by creating an SQL join and
including the fields of the related object in the
SELECT statement.
• For this reason, select_related gets the related
objects in the same database query.
• However, to avoid the much larger result set that
would result from joining across a ‘many’
relationship, select_related is limited to single-valued
relationships - foreign key and one-to-one.

39. List View
def blog_list(request):!
blogs = Blog.objects.all()!
blogs = blogs.select_related("submitter")!
return render(request, "blog/blog_list.html", {!
"blogs": blogs,!
})!

43. Second view:
The blog detail page

51. prefetch_related()
• prefetch_related does a separate lookup for
each relationship, and does the ’joining’ in
Python.
• This allows it to prefetch many-to-many and
many-to-one objects … in addition to the
foreign key and one-to-one relationships.
• It also supports prefetching of GenericRelation
and GenericForeignKey.

52. Detail View
def blog_detail(request, blog_id):!
blog = get_object_or_404(Blog, id=blog_id)!
posts = Post.objects.filter(blog=blog)!
posts = posts.prefetch_related(!
“comments__submitter", "likers",!
)!
return render(request, "blog/blog_detail.html", {!
"blog": blog,!
"posts": posts,!
})!

54. Summary
• The QuerySet API methods select_related() and
prefetch_related() automate some best practices
to avoid extra queries in views/templates.
• Use select_related() for one-to-many or one-to-one
relations.
• Use prefetch_related() for many-to-many or
many-to-one (e.g. reverse foreign key) relations.

55. Thanks!
Christopher Adams (@adamsc64)
https://github.com/adamsc64/a-related-matter