O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Using Recursive Common Table Expressions with Ecto

52 visualizações

Publicada em

Using Recursive Common Table Expressions with Ecto

Publicada em: Tecnologia
  • Seja o primeiro a comentar

  • Seja a primeira pessoa a gostar disto

Using Recursive Common Table Expressions with Ecto

  1. 1. Maarten van Vliet Backend developer @ Awkward Email: maarten@awkward.co Github: maartenvanvliet
  2. 2. Recursive Common Table Expressions and Ecto PRESENTATION By Maarten van Vliet
  3. 3. First: an introduction to the problem
  4. 4. What is Sketch? An intuitive vector editor for the Mac. It’s used primarily by screen designers who create websites, icons, and user interfaces for desktop and mobile devices.
  5. 5. Sketch Cloud Sketch Cloud is a platform that allows you to share documents easily and with everyone. Many more features are coming! Sketch Cloud uses a GraphQL API built in Elixir, we call it SketchQL
  6. 6. Prototyping Sketch’s Prototyping features makes it easy to create interactive workflows and preview your designs as your users will see them. Released last year in Sketch and Sketch Cloud A user can now create a prototype in the Sketch, upload it to Cloud and interactively play with it
  7. 7. Prototyping Cloud Building prototyping was challenging • Fluent transitions across browsers • Converting Sketch Prototypes to the web • And, there are simple prototypes such as this one
  8. 8. And complex prototypes…
  9. 9. Problems We needed to fluently transition from one screen to the next for prototyping in the browser. This meant: (deep) preloading the relations of one screen (artboard) with all other artboards So, when A is loaded, we need to load B and C, but also D!
  10. 10. Simplest solution Recursively query database for related artboards from application 1. First query for artboard A 2. Query for artboards directly related to A, returns [B, C] 3. Query for artboards directly related to [B, C], but leave out already found artboards [A], this returns [D] 4. Query for artboards directly related to [D], but leave out already found artboards [A, B, C], returns [] 5. We stop when an empty set is returned. Problem:lots of queries
  11. 11. Solution: Recursive Common Table Expressions! • Last year we migrated Sketch Cloud to Mariadb 10.2 • Introduced support for (Recursive) Common Table Expressions • (R)CTE's are also available in Mysql 8.0 (since 2018), and Postgres 8.4 (since 2009) • But what are they?
  12. 12. Common Table Expressions A CTE is a temporary resultset Think of it as a database view only created and visible for one query. Useful for making subqueries easier to read You can have multiple in one query WITH FirstUser AS (   SELECT * FROM Users WHERE id = 1 ) SELECT * FROM FirstUser — Equivalent to query with subquery SELECT * FROM (SELECT * FROM Users WHERE id = 1) AS F;
  13. 13. CTE’s can also do recursion! Recursive CTE’s are useful for querying hierarchies, e.g. tables with a parent_id column, so a row has can have a parent or children E.g. a CMS with pages, where a page can have children Pages: WITH RECURSIVE PageGraph AS ( SELECT P.id, P.parent_id FROM Pages P WHERE P.parent_id IS NULL —start id UNION SELECT P.id, P.parent_id FROM Pages P JOIN PageGraph PG ON P.parent_id = PG.id ) SELECT * FROM PageGraph Id parent_id Name 1 NULL Page 1 2 1 Subpage 1 3 1 Subpage 2 4 2 Subpage 3
  14. 14. Dealing with cycles How to deal with cycles? Hierarchies with “loops” in them. E.g. page A has page B as a parent, and page B has page A as a parent WITH RECURSIVE PageGraph AS ( SELECT P.id, P.parent_id FROM Pages P WHERE P.id = 1 #start id UNION SELECT P.id, P.parent_id FROM Pages P JOIN PageGraph PG ON P.parent_id = PG.id ) SELECT * FROM PageGraph Id parent_id Name 1 2 Page 1 2 1 Page 2 Union removes duplicates!
  15. 15. Back to the problem In steps: • First get artboards related to A, and store them in “to”, returns [B, C] • UNION this with the artboards where the id matches those of [B, C] • Get related artboards of [B, C], returns [D] • Again, UNION and get related artboards of [D], returns [A]. • Nothing new found, so stop WITH RECURSIVE RelatedArtboards AS ( SELECT — A.id AS "from", F.DestinationArtboardId AS "to" FROM Artboards A JOIN Layers L ON L.ArtboardId = A.id JOIN Flows F ON F.id = L.FlowId WHERE A.id = #Start ID, in this case Artboard A UNION SELECT — A.id AS "from", F.DestinationArtboardId AS "to" FROM Artboards A JOIN Layers L ON L.ArtboardId = A.id JOIN Flows F ON F.id = L.FlowId JOIN RelatedArtboards ON A.id = RelatedArtboards.to WHERE A.id = RelatedArtboards.to ) SELECT R.to FROM RelatedArtboards R From To A B A C B D C D D A
  16. 16. Now we only need one query to load all artboards for a prototype! But how to use this in Elixir/Ecto?
  17. 17. Not supported in the query builder, yet… Still open 😢
  18. 18. Once merged: page_tree_initial_query = Page |> where([p], is_nil(p.parent_id)) page_tree_recursion_query = Page |> join(:inner, [p], pt in "page_tree", on: p.parent_id == pt.id) page_tree_query = page_tree_initial_queryv |> union(^page_tree_recursion_query) Page |> recursive_ctes(true) |> with_cte("page_tree", as: ^page_tree_query) |> Repo.all
  19. 19. Until then… Fragments gives us the ability to extend Ecto defmacro with_related_artboards(artboard_id) do quote do fragment( """ ( WITH RECURSIVE RelatedArtboards AS ( SELECT F.DestinationArtboardId AS "to" FROM Artboards A JOIN Layers L ON L.ArtboardId = A.id JOIN Flows F ON F.id = L.FlowId WHERE A.id = ? UNION SELECT F.DestinationArtboardId AS "to" FROM Artboards A JOIN Layers L ON L.ArtboardId = A.id JOIN Flows F ON F.id = L.FlowId JOIN RelatedArtboards ON A.id = RelatedArtboards.to WHERE A.id = RelatedArtboards.to ) SELECT RelatedArtboards.to FROM RelatedArtboards WHERE RelatedArtboards.to IS NOT NULL ) """, unquote(artboard_id) ) end end import Sketchql.Utils.RelatedArtboards artboard_id = 1 Artboard |> join(:inner, [a], ra in with_related_artboards(^artboard_id) |> Repo.all() So, this will return a list of %Artboard{} Ecto.Schema structs related to the artboard with id 1. • Keep composability of queries
  20. 20. 🎉 Conclusion • With one query leveraging Ecto and RCTE ’s we can query all artboards related to the current one, no matter how deep. • In the app we also paginate these calls. This way we can render much larger prototypes in Sketch Cloud • It really pays off to dive deep into the tools your database can provide such as RCTE’s. • Ecto’s extensibility is great! Where we could not use its native features we could use SQL to make up for it

×