Quill is a new No-ORM framework that lets you code-generate SQL queries similar to Slick but is more useable, simpler, and more mathematically sound.

1. Slick v.s. Quill Smackdown
Alexander Ioffe

2. Introduction
v.s.

3. • 8 yrs old, 38K Lines
• Embedded DSL (EDSL)
• Supported Databases:
SQLServer,
Oracle,
DB2,
MySQL,
PostgreSQL,
SQLite,
Derby,
HSQLDB,
H2
• 2 yrs old, 34k lines
• Quoted DSL (QDSL)
• Supported Databases:
SQLServer
MySQL,
PostgreSQL,
SQLite,
H2,
Spark,
Cassandra,
OrientDB

4. • Runtime Queries (default),
optional API for Compile-Time
• APIs for Async, Streaming,
Sync (3rd party), and Effect-Type
tracking.
• Largest Contributors:
Stefan Zeiger (170k),
Christopher Vogt (11k),
Hemant Kumar (1.7k)
• Compile Time Queries (default),
automatic fallback to Runtime
• APIs for Sync, Async, and via
Finagle. Only Streaming for
Cassandra.
• Largest Contributors:
Flavio Brasil (79k),
Mykhailo Osypov (5.4k),
Juliano Alves (2.8k),
Michael Ledin (2.7k),
Gustavo Amigo (2.6k),
jilen (2.4k),
Subhobrata Dey (2k),

5. Anatomy of a Slick Query
TableQuery[Person].filter(_.age > 10)
TableQuery[Person].filter(age =>
columnExtensionMethods(person.age).>(LiteralColumn[Int](10))
)((CanBeQueryCondition.BooleanColumnCanBeQueryCondition))
Filter(
from: Table(
PERSON,
Path(NAME), Path(AGE)…
)
where: Apply(
Function >
arg0: Path AGE
arg1: LiteralNode(10)
)
)
Expanded Implicits
Slick AST
select
NAME,
AGE…
from PERSON where AGE > 10
Query
Scala Compiler
~Scala Code
JDBC
Context
Evaluator

6. Anatomy of a Quill Query
quote{ query[Person].filter(_.age > 10) }
EntityQuery[Person]).filter(((x$1: Person) => x$1.age.>(10)))
select
NAME,
AGE…
from PERSON where AGE > 10
(Compile Time) Queries
querySchema("Person").filter(
x1 => x1.age > 10)
.map(x => (x.id, x.name, x.age))
Scala Compiler
Macro Engine
Quasi Quote Parser
Scala AST
Quill AST

7. Which one is Better? Slick Quill
Usability
Reliability
Extension Friendliness
Streaming
Testing
Ecosystem
Bonus

8. Usability
v.s.

9. Previously, on PHASE

10. SELECT DISTINCT
account.name, alias,
CASE WHEN code = 'EV'
THEN cast(account.number AS VARCHAR)
ELSE cast(account.number AS VARCHAR) + substring(alias, 1, 2) END AS OFFICIAL_IDENTITY,
CASE WHEN order_permission IN ('A', 'S')
THEN 'ST' ELSE 'ENH' END
FROM (
SELECT DISTINCT mc.alias, mc.code, mc.order_permission, mc.account_tag
FROM MERCHANT_CLIENTS mc
JOIN REGISTRY r ON r.alias = mc.alias
WHERE r.market = 'us' AND r.record_type = 'M'
UNION ALL
SELECT DISTINCT sc.alias, 'EV' AS code, part.order_permission, sc.account_tag
FROM SERVICE_CLIENTS sc
JOIN REGISTRY r ON r.alias = sc.alias AND r.record_type = 'S' AND r.market = 'us'
JOIN PARTNERSHIPS part ON part.id = sc.partnership_fk) client
INNER JOIN (
dbo.ACCOUNTS account
INNER JOIN ACCOUNT_TYPES accountType ON account.type = accountType.account_type
LEFT JOIN DEDICATED_ACCOUNTS dedicated ON dedicated.account_number = account.number)
ON (accountType.mapping_type = 0)
OR (accountType.mapping_type = 2 AND account.tag = client.account_tag)
OR (accountType.mapping_type = 1 AND dedicated.client_alias = client.alias)

12. SELECT DISTINCT
account.name, alias,
CASE (...) AS OFFICIAL_IDENTITY,
CASE (...)
FROM (
SELECT DISTINCT
mc.alias, mc.code, mc.order_permission, mc.account_tag (code, alias, perm, tag)
FROM MERCHANT_CLIENTS mc
JOIN REGISTRY r ON (alias) AND (otherConditions)
UNION ALL
SELECT DISTINCT
sc.alias, 'EV' AS code, part.order_permission, sc.account_tag (code, alias, perm, tag)
FROM SERVICE_CLIENTS sc
JOIN REGISTRY r ON (alias) AND (otherConditions)
JOIN PARTNERSHIPS part ON (id <-> fk)) client
INNER JOIN (
dbo.ACCOUNTS account
INNER JOIN ACCOUNT_TYPES accountType ON (account_type)
LEFT JOIN DEDICATED_ACCOUNTS dedicated ON (account_number)
)
ON (possibly anything...)
OR (possibly the account tag...)
OR (possibly the alias...) → All Depending on the accountType

13. SELECT DISTINCT
account.name, alias,
CASE (...) AS OFFICIAL_IDENTITY,
CASE (...)
FROM (
SELECT DISTINCT
mc.alias, mc.code, mc.order_permission, mc.account_tag (code, alias, perm, tag)
FROM MERCHANT_CLIENTS mc
JOIN REGISTRY r ON (alias) AND (otherConditions)
UNION ALL
SELECT DISTINCT
sc.alias, 'EV' AS code, part.order_permission, sc.account_tag (code, alias, perm, tag)
FROM SERVICE_CLIENTS sc
JOIN REGISTRY r ON (alias) AND (otherConditions)
JOIN PARTNERSHIPS part ON (id <-> fk)) client
INNER JOIN (
dbo.ACCOUNTS account
INNER JOIN ACCOUNT_TYPES accountType ON (account_type)
LEFT JOIN DEDICATED_ACCOUNTS dedicated ON (account_number)
)
ON (possibly anything...)
OR (possibly the account tag...)
OR (possibly the alias...) → All Depending on the accountType

14. CREATE FUNCTION dbo.merchantClientsUdf (@market)
RETURNS table as RETURN (
SELECT DISTINCT
alias, code, order_permission, account_tag
FROM MERCHANT_CLIENTS merchantClient
JOIN REGISTRY entry
ON entry.alias = merchantClient.alias
WHERE entry.market = @market
AND entry.record_type = 'M')
CREATE VIEW CLIENT_ACCOUNTS AS
SELECT DISTINCT
account.name, alias,
CASE WHEN code = 'EV'
THEN cast(account.number AS VARCHAR)
ELSE cast(account.number AS VARCHAR) + substring(alias, 1, 2) END AS OFFICIAL_IDENTITY,
CASE WHEN order_permission IN ('A', 'S')
THEN 'ST' ELSE 'ENH' END
FROM (select * from merchantClientsUdf ('us') union
select * from serviceClientsUdf ('us')) as client
INNER JOIN (
dbo.ACCOUNTS account
INNER JOIN ACCOUNT_TYPES accountType ON account.type = accountType.account_type
LEFT JOIN DEDICATED_ACCOUNTS dedicated ON dedicated.account_number = account.number)
ON (accountType.mapping_type = 0)
OR (accountType.mapping_type = 2 AND account.tag = client.account_tag)
OR (accountType.mapping_type = 1 AND dedicated.client_alias = client.alias)
CREATE FUNCTION dbo.serviceClientsUdf (@market)
RETURNS table as RETURN (
SELECT DISTINCT
alias, code, order_permission, account_tag
FROM SERVICE_CLIENTS serviceClient
JOIN REGISTRY entry
ON entry.alias = serviceClient.alias
AND entry.record_type = 'S'
AND entry.market = @market
JOIN PARTNERSHIPS partnership
ON partnership.id = serviceClient.partnership_fk)

15. CREATE FUNCTION dbo.merchantClientsUdf (@market)
RETURNS table as RETURN (…)
CREATE VIEW EU_CLIENT_ACCOUNTS AS
SELECT DISTINCT
account.name, alias,
CASE WHEN code = 'EV'
THEN cast(account.number AS VARCHAR)
ELSE cast(account.number AS VARCHAR) + substring(alias, 1, 2) END AS OFFICIAL_IDENTITY,
CASE WHEN order_permission IN ('A', 'S')
THEN 'ST' ELSE 'ENH' END
FROM (select * from merchantClientsUdf ('eu') union
select * from enhancedServiceClientsUdf ('eu')) as client
INNER JOIN (
dbo.ACCOUNTS account
INNER JOIN ACCOUNT_TYPES accountType ON account.type = accountType.account_type
LEFT JOIN DEDICATED_ACCOUNTS dedicated ON dedicated.account_number = account.number)
ON (accountType.mapping_type = 0)
OR (accountType.mapping_type = 2 AND account.tag = client.account_tag)
OR (accountType.mapping_type = 1 AND dedicated.client_alias = client.alias)
CREATE FUNCTION dbo.enhancedServiceClientsUdf (@market)
RETURNS table as RETURN (
SELECT DISTINCT
alias, code, order_permission, account_tag
FROM SERVICE_CLIENTS serviceClient
JOIN REGISTRY entry
ON (... entry.market = @market ...)
JOIN PARTNERSHIPS partnership ON (…)
JOIN PARTNERSHIP_CODES pc
ON partnership.ID = pc.partnership_fk

16. CREATE FUNCTION dbo.merchantClientsUdf (@market)
RETURNS table as RETURN (…)
CREATE VIEW CA_CLIENT_ACCOUNTS AS
SELECT DISTINCT
account.name, alias,
CASE WHEN code = 'EV'
THEN cast(account.number AS VARCHAR)
ELSE cast(account.number AS VARCHAR) + substring(alias, 1, 2) END AS OFFICIAL_IDENTITY,
CASE WHEN order_permission IN ('A', 'S')
THEN 'ST' ELSE 'ENH' END
FROM (select * from merchantClientsUdf ('ca')) as client
INNER JOIN (
dbo.ACCOUNTS account
INNER JOIN ACCOUNT_TYPES accountType ON account.type = accountType.account_type
LEFT JOIN DEDICATED_ACCOUNTS dedicated ON dedicated.account_number = account.number)
ON (accountType.mapping_type = 0)
OR (accountType.mapping_type = 2 AND account.tag = client.account_tag)
OR (accountType.mapping_type = 1 AND dedicated.client_alias = client.alias)

17. ~O(N)/2 Codebase Size Per N Business Units!
Tables:
• merchantClientsUd
f
• serviceClientsUdf
• CLIENT_ACCOUNTS
• merchantClientsUdf
• serviceClientsUdf
• US_CLIENT_ACCOUNTS
• enhancedServiceClientsUdf +
• EU_CLIENT_ACCOUNTS +
• CA_CLIENT_ACCOUNTS +
Tables:
= (Still) Lots of Technical Debt

19. … and behold!

20. def merchantClientsUdf(market:String):Query[(String, String, Char, String)] = {
for {
mc <- merchantClients
r <- registry
if (r.alias === mc.alias && r.market === market
&& r.recordType === 'M')
} yield (mc.alias, mc.code, mc.orderPermission, mc.accountTag)
}
SELECT DISTINCT mc.alias, mc.code, order_permission, mc.account_tag
FROM MERCHANT_CLIENTS mc
JOIN REGISTRY r ON r.alias = mc.alias
WHERE r.market = 'us' AND r.record_type = 'M'

21. def merchantClientsUdf(market:String):Query[(String, String, Char, String)] = {
for {
mc <- merchantClients
r <- registry
if (r.alias === mc.alias && r.market === market
&& r.recordType === 'M')
} yield (mc.alias, mc.code, mc.orderPermission, mc.accountTag)
}
SELECT DISTINCT mc.alias, mc.code, order_permission, mc.account_tag
FROM MERCHANT_CLIENTS mc
JOIN REGISTRY r ON r.alias = mc.alias
WHERE r.market = 'us' AND r.record_type = 'M'
This Is a Lie

22. def merchantClientsUdf(market:String):
Query[(Rep[String],Rep[String],Rep[Char],Rep[String]),
(String,String,Char, String),Seq]=
{
for {
mc <- merchantClients
r <- registry
if (r.alias === mc.alias && r.market === market
&& r.recordType === 'M')
} yield (mc.alias, mc.code, mc.orderPermission, mc.accountTag)
}
SELECT DISTINCT mc.alias, mc.code, order_permission, mc.account_tag
FROM MERCHANT_CLIENTS mc
JOIN REGISTRY r ON r.alias = mc.alias
WHERE r.market = 'us' AND r.record_type = 'M'
The Truth Is

23. def merchantClientsUdf(market:String):
Query[ClientLifted,
Client,Seq] =
{
for {
mc <- merchantClients
r <- registry
if (r.alias === mc.alias && r.market === market
&& r.recordType === 'M')
} yield (mc.alias, mc.code, mc.orderPermission, mc.accountTag)
}
SELECT DISTINCT mc.alias, mc.code, order_permission, mc.account_tag
FROM MERCHANT_CLIENTS mc
JOIN REGISTRY r ON r.alias = mc.alias
WHERE r.market = 'us' AND r.record_type = 'M'
The Truth Is

24. def merchantClientsUdf(market:String) = quote {
for {
mc <- merchantClients
r <- registry
if (r.alias == mc.alias && r.market == lift(market)
&& r.recordType == "M")
} yield (mc.alias, mc.code, mc.orderPermission, mc.accountTag)
}
SELECT DISTINCT mc.alias, mc.code, order_permission, mc.account_tag
FROM MERCHANT_CLIENTS mc
JOIN REGISTRY r ON r.alias = mc.alias
WHERE r.market = 'us' AND r.record_type = 'M'
Quill Is Similar

25. def merchantClientsUdf(market:String):
Quoted[Query[(String,String,Char,String)]]=
quote {
for {
mc <- merchantClients
r <- registry
if (r.alias == mc.alias && r.market == lift(market)
&& r.recordType == "M")
} yield (mc.alias, mc.code, mc.orderPermission, mc.accountTag)
}
SELECT DISTINCT mc.alias, mc.code, order_permission, mc.account_tag
FROM MERCHANT_CLIENTS mc
JOIN REGISTRY r ON r.alias = mc.alias
WHERE r.market = 'us' AND r.record_type = 'M'
… but with sane type signatures

26. def merchantClientsUdf(market:String):
Quoted[Query[(String,String,Char,String)]]=
quote {
for {
mc <- mercantClients
r <- registry
if (r.alias == mc.alias && r.market == lift(market)
&& r.recordType == "M")
} yield (mc.alias, mc.code, mc.orderPermission, mc.accountTag)
}
SELECT DISTINCT mc.alias, mc.code, order_permission, mc.account_tag
FROM MERCHANT_CLIENTS mc
JOIN REGISTRY r ON r.alias = mc.alias
WHERE r.market = 'us' AND r.record_type = 'M'
… but with sane type signatures

27. def merchantClientsUdf(market:String):
Quoted[Query[Client]]=
quote {
for {
mc <- merchantClients
r <- registry
if (r.alias == mc.alias && r.market == lift(market)
&& r.recordType == "M")
}
yield Client(mc.alias, mc.code, mc.orderPermission, mc.accountTag)
}
SELECT DISTINCT mc.alias, mc.code, order_permission, mc.account_tag
FROM MERCHANT_CLIENTS mc
JOIN REGISTRY r ON r.alias = mc.alias
WHERE r.market = 'us' AND r.record_type = 'M'
… with Case Classes it’s even better

28. def merchantClientsUdf(market:String):
Query[Quoted[(Option[String], Option[String], Option[Char], Option[String])]]
= quote {
for {
mc <- merchantClients
r <- registry
if (r.alias == mc.alias && r.market == lift(market)
&& r.recordType == "M")
} yield (mc.alias, mc.code, mc.orderPermission, mc.accountTag)
}
It’s a bit touchy with Optionals

29. … and for a reason
SELECT DISTINCT
mc.alias, mc.code, order_permission,
mc.account_tag
FROM MERCHANT_CLIENTS mc
JOIN REGISTRY r ON r.alias = mc.alias

30. select * from MERCHANT_CLIENTS
where ACCOUNT_TAG = null
select * from MERCHANT_CLIENTS
where ACCOUNT_TAG is null
Always False
Can be True
(null = null) = false
Says:

31. select * from MERCHANT_CLIENTS
where ACCOUNT_TAG = null
select * from MERCHANT_CLIENTS
where ACCOUNT_TAG is null
Always False
Can be True
(null = null) = false

32. select * from MERCHANT_CLIENTS
where ACCOUNT_TAG = null
select * from MERCHANT_CLIENTS
where ACCOUNT_TAG is null
Always False
Can be True
Can be True
(null = null) = true
SET ANSI_NULLS OFF

33. def merchantClientsUdf(market:String):
Query[Quoted[(Option[String], Option[String], Option[Char], Option[String])]]
= quote {
for {
mc <- mercantClients
r <- registry
if (r.alias.exists(rr == mc.alias.exists(_ == rr))
&& r.market.exists(_ == lift(market))
&& r.recordType.exists(_ == "M"))
} yield (mc.alias, mc.code, mc.orderPermission, mc.accountTag)
}
This will solve the problem…

34. ... or Make Some Operators
implicit class OptionalExtensions[T](o:Option[T]) {
def ===(p:Option[T]) = {
quote { o.exists(oo =>p.exists(_ == oo)) }
}
def ~=~(p:Option[T]) = {
quote { o.exists(oo =>p.exists(_ == oo)) }
}
def ~==(p:T) = {
quote { o.exists(_ == p) }
}
}
implicit class PlainExtensions[T](o:T) {
def ==~(p:Option[T]) = {
quote{ p.exists(_ == o) }
}
}

35. def merchantClientsUdf(market:String):
Query[Quoted[(Option[String], Option[String], Option[Char], Option[String])]]
= quote {
for {
mc <- mercantClients
r <- registry
if (r.alias ~=~ mc.alias
&& r.market ~== lift(market)
&& r.recordType ~== "M")
} yield (mc.alias, mc.code, mc.orderPermission, mc.accountTag)
}
… and it can be remedied

36. Overview…

37. def merchantClientsUdf(market:String):Query[ClientLifted, Client, Seq] =
for {
mc <- mercantClients
r <- registry
if (r.alias === mc.alias && r.market === market && r.recordType === 'M')
} yield Client(mc.alias, mc.code, mc.orderPermission, mc.accountTag)
def serviceClientsUdf(market:String):Query[ClientLifted, Client, Seq] =
for {
sc <- serviceClients
r <- registry
if (r.alias === sc.alias && r.market === market && r.recordType === 'S')
part <- parnerships
if (part.id === sc.partnershipFk)
} yield Client(sc.alias, "EV".bind.?, part.orderPermission, sc.accountTag)
def clients(market:String):Query[ClientLifted, Client, Seq]
= merchantClientsUdf(market) ++ serviceClientsUdf(market)

38. def merchantClientsUdf(market:String):Quoted[Query[Client]] = quote {
for {
mc <- mercantClients
r <- registry
if (r.alias ~=~ mc.alias) && (r.market ~=~ lift(market)) && (r.recordType ~== "M")
} yield Client(mc.alias, mc.code, mc.orderPermission, mc.accountTag)
}
def serviceClientsUdf(market:String):Quoted[Query[Client]] = quote {
for {
sc <- serviceClients
r <- registry
if (r.alias ~=~ sc.alias) && (r.market ~== lift(market)) && (r.recordType ~== "S")
part <- parnerships
if (part.id == sc.partnershipFk)
} yield Client(sc.alias, Some("EV"), part.orderPermission, sc.accountTag)
}
def clients(market:String):Quoted[Query[Client]] = quote {
merchantClientsUdf(market) ++ serviceClientsUdf(market)
}

39. Stepping
Back

40. SELECT DISTINCT
account.name, alias,
CASE (...) AS OFFICIAL_IDENTITY,
CASE (...)
FROM (...) client
INNER JOIN (
dbo.ACCOUNTS account
INNER JOIN ACCOUNT_TYPES accountType ON (account_type)
LEFT JOIN DEDICATED_ACCOUNTS dedicated ON (account_number)
)
ON (accountType.mapping_type = 0)
OR (possibly the account tag...)
OR (possibly the alias...)
name alias OFFICIAL_IDENTITY perm
TUNV FNF 111 ENH
TUNV ACME 111AC ENH
SIADV FNF 456 ENH
AUNV FNF 222 ENH
AUNV ACME 222AC ENH
ACMEINV ACME 808AC ENH
YOGADV YOGL 123 ST
TUNV YOGL 111 ST
AUNV YOGL 222 ST

41. SELECT DISTINCT
account.name, alias,
CASE (...) AS OFFICIAL_IDENTITY,
CASE (...)
FROM (...) client
INNER JOIN (
dbo.ACCOUNTS account
INNER JOIN ACCOUNT_TYPES accountType ON (account_type)
LEFT JOIN DEDICATED_ACCOUNTS dedicated ON (account_number)
)
ON (possibly anything...)
OR (accountType.mapping_type = 2 AND account.tag = client.account_tag)
OR (possibly the alias...)
name alias OFFICIAL_IDENTITY perm
TUNV FNF 111 ENH
TUNV ACME 111AC ENH
SIADV FNF 456 ENH
AUNV FNF 222 ENH
AUNV ACME 222AC ENH
ACMEINV ACME 808AC ENH
YOGADV YOGL 123 ST
TUNV YOGL 111 ST
AUNV YOGL 222 ST

42. SELECT DISTINCT
account.name, alias,
CASE (...) AS OFFICIAL_IDENTITY,
CASE (...)
FROM (...) client
INNER JOIN (
dbo.ACCOUNTS account
INNER JOIN ACCOUNT_TYPES accountType ON (account_type)
LEFT JOIN DEDICATED_ACCOUNTS dedicated ON (account_number)
)
ON (possibly anything...)
OR (possibly the account tag...)
OR (accountType.mapping_type = 1 AND dedicated.client_alias = client.alias)
name alias OFFICIAL_IDENTITY perm
TUNV FNF 111 ENH
TUNV ACME 111AC ENH
SIADV FNF 456 ENH
AUNV FNF 222 ENH
AUNV ACME 222AC ENH
ACMEINV ACME 808AC ENH
YOGADV YOGL 123 ST
TUNV YOGL 111 ST
AUNV YOGL 222 ST

43. def mappingConditionsMet(
mappingType: Rep[Int],
accountTag: Rep[String],
clientTag: Rep[String],
clientAlias: Rep[String],
dedicatedAlias: Rep[String]
):Rep[Int] =
Case.If(mappingType === 0).Then(1)
.If(mappingType === 2 && accountTag === clientTag).Then(1)
.If(mappingType === 1 && clientAlias === dedicatedAlias).Then(1)
.Else(0)
def accountMapping(clients:Query[Client]):
Query[
ClientsLifted,AccountsLifted,AccountTypesLifted,Option[Rep[DedicatedAccounts]]
Clients,Accounts,AccountTypes,Option[DedicatedAccounts],
Seq
] = {
for {
(account, accountType, dedicatedAccount) <- allAccounts
client <- clients if (mappingConditionsMet(
accountType.mappingType.getOrElse(0), account.tag.getOrElse(""),
client.accountTag.getOrElse(""), client.alias.getOrElse(""),
dedicatedAccount.map(_.clientAlias).flatten.getOrElse("")) == 1)
} yield (client, account, accountType, dedicatedAccount)
}

44. val mappingConditionsMet = quote {
(
mappingType: Option[Int],
accountTag: Option[String],
clientTag: Option[String],
clientAlias: Option[String],
dedicatedAlias: Option[Option[String]]
) =>
if (mappingType == 0) 1
else if ((mappingType == 2) && (accountTag ==~ clientTag)) 1
else if ((mappingType == 1) && (dedicatedAlias.exists(_ ~=~ clientAlias))) 1
else 0
}
def accountMapping(clients:Quoted[Query[Client]]):
Quoted[Query[(Client, Accounts, AccountTypes, Option[DedicatedAccounts])]] =
quote {
for {
(account, accountType, dedicatedAccount) <- allAccounts
client <- clients if (
mappingConditionsMet(
accountType.mappingType, account.tag,
client.accountTag, client.otherAlias,
dedicatedAccount.map(_.clientAlias)
) == 1)
} yield (client, account, accountType, dedicatedAccount)
}

45. val mappingConditionsMet:
Quoted[(Int, String, Option[String], Option[String], Option[Option[String]]) => Int] = quote {
(
mappingType: Option[Int],
accountTag: Option[String],
clientTag: Option[String],
clientAlias: Option[String],
dedicatedAlias: Option[Option[String]]
) =>
if (mappingType == 0) 1
else if ((mappingType == 2) && (accountTag ==~ clientTag)) 1
else if ((mappingType == 1) && (dedicatedAlias.exists(_ ~=~ clientAlias))) 1
else 0
}
def accountMapping(clients:Quoted[Query[Client]]):
Quoted[Query[(Client, Accounts, AccountTypes, Option[DedicatedAccounts])]] =
quote {
for {
(account, accountType, dedicatedAccount) <- allAccounts
client <- clients if (
mappingConditionsMet(
accountType.mappingType, account.tag,
client.accountTag, client.otherAlias,
dedicatedAccount.map(_.clientAlias)
) == 1)
} yield (client, account, accountType, dedicatedAccount)
}

46. Actions
DBIO.seq(
TableQuery[Person] += Person("Joe", "Roe")
)
DBIO.seq(
TableQuery[Person] ++=
Seq(Person("Joe", "Roe"), ...)
)
quote {
query[Person].insert(lift(Person("Joe", "Roe")))
}
quote {
liftQuery(List(Person("Joe", "Roe")), …)
.foreach(e => query[Person].insert(e))
}
Individual
Bulk

47. Actions Continued…
DBIO.seq(
TableQuery[Person].map(_.firstName) += ("Joe")
)
quote {
query[Person].insert(_.firstName -> lift("Joe"))
}
Insert
Specific
Columns

48. Actions Continued…
DBIO.seq(
(TableQuery[Person].returning(person.id)) += Record(0, "1")
)
DBIO.seq(
(TableQuery[Person] ].returning(person.id)) ++=
Seq(Person("Joe", "Roe"), ...)
)
quote {
query[Person].insert(lift(Record(0, "1"))).returning(_.id)
}
quote {
liftQuery(List(Record(0, "1")), …)
.foreach(e => query[Person].insert(e).returning(_.id))
}
Inserting
Returning Ids
Individual
Inserting
Returning Ids
Individual

49. Show me the Queries

50. SELECT
s189.s137, s189.s138, s189.s139, s189.s140, s176."NAME", s176."TAG", s176."NUMBER", s176."TYPE",
s177."ACCOUNT_TYPE", s177."MAPPING_TYPE", s47.s118, s47.s119, s47.s120
FROM "ACCOUNTS" s176
INNER JOIN "ACCOUNT_TYPES" s177
ON s176."TYPE" = s177."ACCOUNT_TYPE"
LEFT OUTER JOIN (
SELECT 1 AS s118, "ACCOUNT_NUMBER" AS s119, "CLIENT_ALIAS" AS s120
FROM "DEDICATED_ACCOUNTS") s47 ON s176."NUMBER" = s47.s119
INNER JOIN (
SELECT s179."ALIAS" AS s137, ? AS s138, s183."ORDER_PERMISSION" AS s139, s179."ACCOUNT_TAG" AS s140
FROM "SERVICE_CLIENTS" s179, "REGISTRY" s180, "PARTNERSHIPS" s183
WHERE ((((CASE WHEN (s179."ALIAS" IS NULL)
THEN ? ELSE cast(s179."ALIAS" AS VARCHAR(255)) END) = s180."ALIAS")
AND (s180."RECORD_TYPE" = 'M')) AND (s180."MARKET" = 'us')) AND (s183."ID" = s179."PARTNERSHIP_FK")
UNION ALL
SELECT s185."ALIAS" AS s137, s185."CODE" AS s138, s185."ORDER_PERMISSION" AS s139, s185."ACCOUNT_TAG" AS s140
FROM "MERCHANT_CLIENTS" s185, "REGISTRY" s186
WHERE (((CASE WHEN (s185."ALIAS" IS NULL)
THEN ? ELSE cast(s185."ALIAS" AS VARCHAR(255)) END) = s186."ALIAS") AND (s186."RECORD_TYPE" = 'M')) AND (s186."MARKET" = 'us'))
s189
ON (CASE WHEN ((CASE WHEN (s177."MAPPING_TYPE" IS NULL)
THEN ? ELSE cast(s177."MAPPING_TYPE" AS INTEGER) END) = 0)
THEN 1
WHEN (((CASE WHEN (s177."MAPPING_TYPE" IS NULL)
THEN ? ELSE cast(s177."MAPPING_TYPE" AS INTEGER) END) = 2) AND ((CASE WHEN (s176."TAG" IS NULL)
THEN ? ELSE cast(s176."TAG" AS VARCHAR(255)) END) = (CASE WHEN (s189.s140 IS NULL)
THEN ? ELSE cast(s189.s140 AS VARCHAR(255)) END)))
THEN 1
WHEN (((CASE WHEN (s177."MAPPING_TYPE" IS NULL)
THEN ? ELSE cast(s177."MAPPING_TYPE" AS INTEGER) END) = 1) AND ((CASE WHEN (s189.s137 IS NULL)
THEN ? ELSE cast(s189.s137 AS VARCHAR(255)) END) = (CASE WHEN ((CASE WHEN (s47.s118 IS NOT NULL)
THEN s47.s120 ELSE NULL END) IS NULL)
THEN ? ELSE cast((CASE WHEN (s47.s118 IS NOT NULL) THEN s47.s120
ELSE NULL END) AS VARCHAR(255)) END)))
THEN 1
ELSE 0 END) = 1

51. SELECT
s189.s137, s189.s138, s189.s139, s189.s140, s176."NAME", s176."TAG", s176."NUMBER", s176."TYPE",
s177."ACCOUNT_TYPE", s177."MAPPING_TYPE", s47.s118, s47.s119, s47.s120
FROM "ACCOUNTS" s176
INNER JOIN "ACCOUNT_TYPES" s177
ON s176."TYPE" = s177."ACCOUNT_TYPE"
LEFT OUTER JOIN (
SELECT 1 AS s118, "ACCOUNT_NUMBER" AS s119, "CLIENT_ALIAS" AS s120
FROM "DEDICATED_ACCOUNTS") s47 ON s176."NUMBER" = s47.s119
INNER JOIN (
SELECT s179."ALIAS" AS s137, ? AS s138, s183."ORDER_PERMISSION" AS s139, s179."ACCOUNT_TAG" AS s140
FROM "SERVICE_CLIENTS" s179, "REGISTRY" s180, "PARTNERSHIPS" s183
WHERE ((((CASE WHEN (s179."ALIAS" IS NULL)
THEN ? ELSE cast(s179."ALIAS" AS VARCHAR(255)) END) = s180."ALIAS")
AND (s180."RECORD_TYPE" = 'M')) AND (s180."MARKET" = 'us')) AND (s183."ID" = s179."PARTNERSHIP_FK")
UNION ALL
SELECT s185."ALIAS" AS s137, s185."CODE" AS s138, s185."ORDER_PERMISSION" AS s139, s185."ACCOUNT_TAG" AS s140
FROM "MERCHANT_CLIENTS" s185, "REGISTRY" s186
WHERE (((CASE WHEN (s185."ALIAS" IS NULL)
THEN ? ELSE cast(s185."ALIAS" AS VARCHAR(255)) END) = s186."ALIAS") AND (s186."RECORD_TYPE" = 'M')) AND (s186."MARKET" = 'us'))
s189
ON (CASE WHEN ((CASE WHEN (s177."MAPPING_TYPE" IS NULL)
THEN ? ELSE cast(s177."MAPPING_TYPE" AS INTEGER) END) = 0)
THEN 1
WHEN (((CASE WHEN (s177."MAPPING_TYPE" IS NULL)
THEN ? ELSE cast(s177."MAPPING_TYPE" AS INTEGER) END) = 2) AND ((CASE WHEN (s176."TAG" IS NULL)
THEN ? ELSE cast(s176."TAG" AS VARCHAR(255)) END) = (CASE WHEN (s189.s140 IS NULL)
THEN ? ELSE cast(s189.s140 AS VARCHAR(255)) END)))
THEN 1
WHEN (((CASE WHEN (s177."MAPPING_TYPE" IS NULL)
THEN ? ELSE cast(s177."MAPPING_TYPE" AS INTEGER) END) = 1) AND ((CASE WHEN (s189.s137 IS NULL)
THEN ? ELSE cast(s189.s137 AS VARCHAR(255)) END) = (CASE WHEN ((CASE WHEN (s47.s118 IS NOT NULL)
THEN s47.s120 ELSE NULL END) IS NULL)
THEN ? ELSE cast((CASE WHEN (s47.s118 IS NOT NULL) THEN s47.s120
ELSE NULL END) AS VARCHAR(255)) END)))
THEN 1
ELSE 0 END) = 1

52. SELECT
client.other_alias, client.code, client.order_permission, client.account_tag,
account_type.name, account_type.tag, account_type.number, account_type.type, account_type.account_type,
account_type.mapping_type,
x11.account_number, x11.client_alias
FROM (SELECT
account.type type, account.name name, account.number number,
account.tag tag, account_type.account_type account_type, account_type.mapping_type mapping_type
FROM accounts account, account_types account_type
WHERE account.type = account_type.account_type) account_type
LEFT JOIN dedicated_accounts x11
ON x11.account_number = account_type.number,
(
(SELECT sc.account_tag account_tag, sc.alias other_alias, ? code, part.order_permission order_permission
FROM service_clients sc, registry r, partnerships part
WHERE sc.alias = r.alias AND r.market = ? AND r.record_type = 'S' AND sc.partnership_fk = part.id)
UNION ALL
(SELECT mc.account_tag account_tag, mc.alias other_alias, mc.code code, mc.order_permission order_permission
FROM merchant_clients mc, registry r1
WHERE mc.alias = r1.alias AND r1.market = ? AND r1.record_type = 'M')
) client
WHERE CASE WHEN CASE WHEN account_type.mapping_type IS NOT NULL
THEN ? ELSE 0 END = 0
THEN 1
WHEN CASE WHEN account_type.mapping_type IS NOT NULL
THEN ?
ELSE 0 END = 2 AND client.account_tag = CASE WHEN account_type.tag IS NOT NULL
THEN ? ELSE '' END
THEN 1
WHEN CASE WHEN account_type.mapping_type IS NOT NULL
THEN ?
ELSE 0 END = 1 AND client.other_alias = x11.client_alias
THEN 1
ELSE 0 END = 1

53. SELECT
client.other_alias, client.code, client.order_permission, client.account_tag,
account_type.name, account_type.tag, account_type.number, account_type.type, account_type.account_type,
account_type.mapping_type,
x11.account_number, x11.client_alias
FROM (SELECT
account.type type, account.name name, account.number number,
account.tag tag, account_type.account_type account_type, account_type.mapping_type mapping_type
FROM accounts account, account_types account_type
WHERE account.type = account_type.account_type) account_type
LEFT JOIN dedicated_accounts x11
ON x11.account_number = account_type.number,
(
(SELECT sc.account_tag account_tag, sc.alias other_alias, ? code, part.order_permission order_permission
FROM service_clients sc, registry r, partnerships part
WHERE sc.alias = r.alias AND r.market = ? AND r.record_type = 'S' AND sc.partnership_fk = part.id)
UNION ALL
(SELECT mc.account_tag account_tag, mc.alias other_alias, mc.code code, mc.order_permission order_permission
FROM merchant_clients mc, registry r1
WHERE mc.alias = r1.alias AND r1.market = ? AND r1.record_type = 'M')
) client
WHERE CASE WHEN CASE WHEN account_type.mapping_type IS NOT NULL
THEN ? ELSE 0 END = 0
THEN 1
WHEN CASE WHEN account_type.mapping_type IS NOT NULL
THEN ?
ELSE 0 END = 2 AND client.account_tag = CASE WHEN account_type.tag IS NOT NULL
THEN ? ELSE '' END
THEN 1
WHEN CASE WHEN account_type.mapping_type IS NOT NULL
THEN ?
ELSE 0 END = 1 AND client.other_alias = x11.client_alias
THEN 1
ELSE 0 END = 1

54. A ‘Slight’ Footnote

55. Usability
Wins!
Obvious Question: Why not just get rid of U

56. Reliability
v.s.

57. Reliability – What are we
measuring?

58. Reliability – What are we
measuring?
Performance Under Load?

59. Reliability – What are we
measuring?
Performance Under Load?
Percentage Code Tested?

60. Reliability – What are we
measuring?
Performance Under Load?
Percentage Code Tested?

61. Reliability – What are we
measuring?
Performance Under Load?
Percentage Code Tested?
Generating Queries Correctly

65. @Entity
public class A{
@OneToMany
@JoinColumn(name= "fk")
private List<B> bs= new ArrayList<B>();
}
for(B b :a.bs) {
doSomethingWith(b)
}

66. @Entity
public class A{
@OneToMany
@JoinColumn(name= "fk")
private List<B> bs= new ArrayList<B>();
}
for(B b :a.bs) {
doSomethingWith(b)
}

69. EH Cache

70. EH Cache

71. EH Cache
JPQL

79. [10.11.2015 15:27:21.088] [ERROR]
[application-akka.actor.default-dispatcher-12]
[application] Error 'internalError - Cannot convertnode to SQLComprehension
| GroupBy t9 : Vector[(t9<String'>, Vector[t17<{s18: String'}>])]
| from s8: Bind :Vector[t17<{s18: String'}>]
| from s13: Table test_query :Vector[@t11<{id: String'}>]
| select: Pure t17 :Vector[t17<{s18: String'}>]
| value:StructNode : {s18: String'}
| s18: Path s13.id : String'
| by: Path s8.s18 : String'
'occured.

80. [10.11.2015 15:27:21.088] [ERROR]
[application-akka.actor.default-dispatcher-12]
[application] Error 'internalError - Cannot convertnode to SQLComprehension
| GroupBy t9 : Vector[(t9<String'>, Vector[t17<{s18: String'}>])]
| from s8: Bind :Vector[t17<{s18: String'}>]
| from s13: Table test_query :Vector[@t11<{id: String'}>]
| select: Pure t17 :Vector[t17<{s18: String'}>]
| value:StructNode : {s18: String'}
| s18: Path s13.id : String'
| by: Path s8.s18 : String'
'occured.

81. [10.11.2015 15:27:21.088] [ERROR]
[application-akka.actor.default-dispatcher-12]
[application] Error 'internalError - Cannot convertnode to SQLComprehension
| GroupBy t9 : Vector[(t9<String'>, Vector[t17<{s18: String'}>])]
| from s8: Bind :Vector[t17<{s18: String'}>]
| from s13: Table test_query :Vector[@t11<{id: String'}>]
| select: Pure t17 :Vector[t17<{s18: String'}>]
| value:StructNode : {s18: String'}
| s18: Path s13.id : String'
| by: Path s8.s18 : String'
'occured.
table.groupBy(_.id).map { case (c, tbl) =>
(c, tbl.length)
}

82. [10.11.2015 15:27:21.088] [ERROR]
[application-akka.actor.default-dispatcher-12]
[application] Error 'internalError - Cannot convertnode to SQLComprehension
| GroupBy t9 : Vector[(t9<String'>, Vector[t17<{s18: String'}>])]
| from s8: Bind :Vector[t17<{s18: String'}>]
| from s13: Table test_query :Vector[@t11<{id: String'}>]
| select: Pure t17 :Vector[t17<{s18: String'}>]
| value:StructNode : {s18: String'}
| s18: Path s13.id : String'
| by: Path s8.s18 : String'
'occured.
table.groupBy(_.id).map { case (c, tbl) =>
(c, tbl.length)
}

83. [10.11.2015 15:27:21.088] [ERROR]
[application-akka.actor.default-dispatcher-12]
[application] Error 'internalError - Cannot convertnode to SQLComprehension
| GroupBy t9 : Vector[(t9<String'>, Vector[t17<{s18: String'}>])]
| from s8: Bind :Vector[t17<{s18: String'}>]
| from s13: Table test_query :Vector[@t11<{id: String'}>]
| select: Puret17 :Vector[t17<{s18: String'}>]
| value:StructNode : {s18: String'}
| s18: Path s13.id : String'
| by: Path s8.s18 : String'
'occured.
table.groupBy(_.id).map { case (c, tbl) =>
(c, tbl.length)
}

84. Table
.groupBy(_.id)
.map { case (c, tbl) =>
(c, tbl.length)
}

85. Table
.groupBy(_.id)
.map { case (c, tbl) =>
(c, tbl.length)
}

86. Table
.drop(0)
.groupBy(_.id)
.map { case (c, tbl) =>
(c, tbl.length)
}

87. … and the solution is:

90. Slick Compiler
Phases:

91. Slick Compiler
Phases:

92. $ git log –p slick/src/main/scala/slick/compiler | awk ... | sort –rn
Total Adds Deletes
2933 5166 2233 Stefan Zeiger
50 50 0 Alexander Ioffe
14 35 21 deusaquilus
7 11 4 Sue
1 12 11 Iulian Dogariu
1 7 6 Ashutosh Agarwal
0 4 4 Mateusz Kołodziejczyk
-16 5 21 Ólafur Páll Geirsson
Who Understands
slick/src/main/scala/slick/compiler
???

93. $ git log –p slick/src/main/scala/slick/compiler | awk ... | sort –rn
Who Understands
slick/src/main/scala/slick/compiler
???
Total Adds Deletes
97.1% 97.7% 97.1% Stefan Zeiger
1.7% 0.9% 0.0% Alexander Ioffe
0.5% 0.7% 0.9% deusaquilus
0.2% 0.2% 0.2% Sue
0.0% 0.2% 0.5% Iulian Dogariu
0.0% 0.1% 0.3% Ashutosh Agarwal
0.0% 0.1% 0.2% Mateusz Kołodziejczyk
0.5% 0.1% 0.9% Ólafur Páll Geirsson

94. $ cat file.txt | awk '/./ && !author { author = $0; next } author {
ins[author] += $1; del[author] += $2 } /^$/ { author = ""; next } END {
for (a in ins) { printf "%10d %10d %10d %sn", ins[a] - del[a], ins[a],
del[a], a } }' | sort –rn
Total Adds Deletes
2933 5166 2233 Stefan Zeiger
50 50 0 Alexander Ioffe
14 35 21 deusaquilus
7 11 4 Sue
1 12 11 Iulian Dogariu
1 7 6 Ashutosh Agarwal
0 4 4 Mateusz Kołodziejczyk
-16 5 21 Ólafur Páll Geirsson
Who Understands
slick/src/main/scala/slick/compiler
???

95. $ cat file.txt | awk '/./ && !author { author = $0; next } author {
ins[author] += $1; del[author] += $2 } /^$/ { author = ""; next } END {
for (a in ins) { printf "%10d %10d %10d %sn", ins[a] - del[a], ins[a],
del[a], a } }' | sort –rn
Total Adds Deletes
2933 5166 2233 Stefan Zeiger
50 50 0 Alexander Ioffe
14 35 21 deusaquilus
7 11 4 Sue
1 12 11 Iulian Dogariu
1 7 6 Ashutosh Agarwal
0 4 4 Mateusz Kołodziejczyk
-16 5 21 Ólafur Páll Geirsson
Who Understands
slick/src/main/scala/slick/compiler
???

96. Enter the Wadler

99. We require that each query in the host language generate
exactly one SQL query. Alluding to twin perils Odysseus
sought to skirt when navigating the straits of Medina, we
seek to avoid Scylla and Charybdis. Scylla stands for the case
where the system fails to generate a query, signalling an error.
Charybdis stands for the case where the system generates
multiple queries, hindering efficiency. The overhead of
accessing a database is high, and to a first approximation cost
is proportional to the number of queries. We particularly want
to avoid a query avalanche, in the sense of Grust et al.
(2010), where a single host query generates a number of SQL
queries proportional to the size of the data

100. Our work avoids these perils. For T-LINQ, we prove the
Scylla and Charybdis theorem, characterising when a host
query is guaranteed to generate a single SQL query. All our
examples are easily seen to satisfy the characterisation in the
theorem, and indeed our theory yields the same SQL query
for each that one would write by hand. For P-LINQ, we
verify that its run time on our examples is comparable to that
of F# 2.0 and F# 3.0, in the cases where those systems
generate a query, and significantly faster in the one case
where F# 3.0 generates an avalanche—indeed, arbitrarily
faster as the size of the data grows.

101. Our work avoids these perils. For T-LINQ, we prove the
Scylla and Charybdis theorem, characterising when a host
query is guaranteed to generate a single SQL query. All our
examples are easily seen to satisfy the characterisation in the
theorem, and indeed our theory yields the same SQL query
for each that one would write by hand. For P-LINQ, we
verify that its run time on our examples is comparable to that
of F# 2.0 and F# 3.0, in the cases where those systems
generate a query, and significantly faster in the one case
where F# 3.0 generates an avalanche—indeed, arbitrarily
faster as the size of the data grows.

105. Enter Flavio

109. It goes without saying…
.length after groupBy + map works
object TestQuery{
valq= quote{
query[TestQuery].groupBy(_.id).map{case(c,tbl) =>
(c, tbl.size)
}
}
defrunQuery= run(q)
}

111. Reliability
Wins!
Obvious Question: Why not just rewrite Slick’s query compiler using Wadler’s Rules

112. Extension Friendliness
v.s.

113. Extension Friendliness
v.s.
Queries

114. def countBy[E, U, K, T](query:Query[E, U, Seq])(predicate:E=>K)(
implicit kshape: Shape[_ <: FlatShapeLevel, K, T, K],
vshape: Shape[_ <: FlatShapeLevel, E, _, E]):
Query[(K, Rep[Int]), (T, Int), scala.Seq] =
{
query.groupBy(predicate).map {
case(field, records) => (field, records.length) }
}
val q = countBy(accounts)(_.`type`)
def conditionalTake[E, U, T <% Ordered](
query:Query[E, U, Seq], numRecords:Option[Int]):
Query[E, U, Seq] = {
numRecords match {
case Some(number) => query.take(number)
case None => query
}
}
val q = conditionalTake(accounts, Some(10))
type count
ADVERTISING 4
INVENTORY 3
TAX 1
http://host/query?nr=100
NAME TAG NUMBER TYPE
TUNV NULL 111 TAX
YOGADV YOG 123 ADVERTISING
SIADV SID 456 ADVERTISING
SIADVA SIDA 457 ADVERTISING
UMBINV NULL 707 INVENTORY
FFADV FF 789 ADVERTISING
ACMEINV NULL 808 INVENTORY
YOGINV NULL 909 INVENTORY

115. def countBy[E, U, K, T](query:Query[E, U, Seq])(predicate:E=>K)(
implicit kshape: Shape[_ <: FlatShapeLevel, K, T, K],
vshape: Shape[_ <: FlatShapeLevel, E, _, E]):
Query[(K, Rep[Int]), (T, Int), scala.Seq] =
{
query.groupBy(predicate).map {
case(field, records) => (field, records.length) }
}
val q = countBy(accounts)(_.`type`)
def conditionalTake[E, U, T <% Ordered](
query:Query[E, U, Seq], numRecords:Option[Int]):
Query[E, U, Seq] = {
numRecords match {
case Some(number) => query.take(number)
case None => query
}
}
val q = conditionalTake(accounts, Some(10))
http://host/query?nr=100
NAME TAG NUMBER TYPE
TUNV NULL 111 TAX
YOGADV YOG 123 ADVERTISING
SIADV SID 456 ADVERTISING
SIADVA SIDA 457 ADVERTISING
UMBINV NULL 707 INVENTORY
FFADV FF 789 ADVERTISING
ACMEINV NULL 808 INVENTORY
YOGINV NULL 909 INVENTORY
type count
ADVERTISING 4
INVENTORY 3
TAX 1

116. def conditionalTake[E](query:Quoted[Query[E]], take:Option[Int]) =
take match {
case Some(num) => quote { query.take(lift(num)) }
case None => quote { query }
}
val q = quote {conditionalTake(accounts)(_.`type`) }
def countBy[E, K] =
quote {
(query:Query[E]) => (predicate: E=>K) =>
query.groupBy(e => predicate(e)).map {
case (field, records) => (field, records.size) }
}
val q = quote { countBy(accounts)(_.`type`) }

117. def conditionalTake[E](query:Quoted[Query[E]], take:Option[Int]) =
take match {
case Some(num) => quote { query.take(lift(num)) }
case None => quote { query }
}
val q = quote { countBy(accounts)(Some(10)) }
def countBy[E, K] =
quote {
(query:Query[E]) => (predicate: E=>K) =>
query.groupBy(e => predicate(e)).map {
case (field, records) => (field, records.size) }
}
val q = quote { countByAlt(accounts)(_.`type`) }
Also takes Quoted[Query[E]] ... but IDEs don’t always understand that

118. Extension Friendliness
Custom Outputs
(from yields, maps, etc...)
v.s.
Queries

119. trait CustomerDef { this: ProfileComponent =>
import profile.api._
case class Client(alias: Rep[Option[String]],
code: Rep[Option[String]], permission: Rep[Option[Char]], tag: Rep[Option[String]])
case class ClientRow(alias: Option[String],
code: Option[String], permission: Option[Char], tag: Option[String])
implicit object ClientRecordShape extends CaseClassShape[
Product,
(Rep[Option[String]], Rep[Option[String]], Rep[Option[Char]], Rep[Option[String]]), Client,
(Option[String], Option[String], Option[Char], Option[String]), ClientRow](Client.tupled,
ClientRow.tupled)
}
Client (alias: Option[String], code: Option[String],
permission: Option[String], tag: Option[String])

120. Structural Type-Based Extensions
case class ShippingOrder(customerId:Int, startedAt:LocalDateTime, endedAt:LocalDateTime)
case class ProcessRequest(regionId:Int, startedAt:LocalDateTime, endedAt:LocalDateTime)
case class Event(typeCode:String, startedAt:LocalDateTime, endedAt:LocalDateTime)

121. Structural Type-Based Extensions
case class ShippingOrder(customerId:Int, startedAt:LocalDateTime, endedAt:LocalDateTime)
case class ProcessRequest(regionId:Int, startedAt:LocalDateTime, endedAt:LocalDateTime)
case class Event(typeCode:String, startedAt:LocalDateTime, endedAt:LocalDateTime)
implicit class TemporalObjectExtensions[T <: { def startedAt: LocalDateTime; def endedAt: LocalDateTime }](
records:Query[T])
{
def existedAt(date: LocalDateTime) =
quote {
records.filter(r => (lift(r.startedAt) < date) && lift(r.endedAt) > date))
}
}
val q = quote { query[ShippingOrder].existedAt(lift(now)) }
val q = quote { query[ProcessRequest].existedAt(lift(now)) }
val q = quote { query[Event].existedAt(lift(now)) }

122. No Infix Date Ops?
case class ShippingOrder(customerId:Int, startedAt:LocalDateTime, endedAt:LocalDateTime)
case class ProcessRequest(regionId:Int, startedAt:LocalDateTime, endedAt:LocalDateTime)
case class Event(typeCode:String, startedAt:LocalDateTime, endedAt:LocalDateTime)
implicit class TemporalObjectExtensions[T <: { def startedAt: LocalDateTime; def endedAt: LocalDateTime }](
records:Query[T])
{
def existedAt(date: LocalDateTime) =
quote {
records.filter(r => (lift(r.startedAt) < date) && lift(r.endedAt) > date))
}
}
val q = quote { query[ShippingOrder].existedAt(lift(now)) }
val q = quote { query[ProcessRequest].existedAt(lift(now)) }
val q = quote { query[Event].existedAt(lift(now)) }

123. No Problem!
implicit classLocalDateOps(left:LocalDateTime){
def> (right:LocalDateTime)= quote(infix"$left >$right".as[Boolean])
def>= (right:LocalDateTime)= quote(infix"$left>=$right".as[Boolean])
def < (right:LocalDateTime)= quote(infix"$left <$right".as[Boolean])
def<= (right:LocalDateTime)= quote(infix"$left<=$right".as[Boolean])
}

124. Extension Friendliness
Wins!
Obvious Question: (Again) Why not just get rid of U

125. Streaming API
v.s.

126. Our Query Comes out…
def doPublish(ds:DataSource) = {
val pub: DatabasePublisher[_ <: Product] =
ds.stream(TableQuery[Accounts]result)
.mapResult(r => serializeToJson(r))
var akkaSource =
Source.fromPublisher(pub)
.map(r => ChunkStreamPart(r))
HttpResponse(entity =
HttpEntity.Chunked(format.contentType, akkaSource)
)
}

127. … of Here!
val route =
get {
pathPrefix("path" / "from" / "host") {
pathEndOrSingleSlash {
ctx => ctx.complete { doPublish(ds) }
}
}
}

128. … of Here!
val route =
get {
pathPrefix("path" / "from" / "host") {
pathEndOrSingleSlash {
ctx => ctx.complete { doPublish(ds) }
}
}
}

129. Streaming
Wins!
Obvious Question: Write Akka Extensions for Quill
… or just move the Monix-based API to Quill-Jdbc from Quill-Cassandra?

130. Testing
v.s.

131. What if we can test Queries…
class MyMemoryDriver extends ModifiedMemoryProfile { }
class MyHeapDriver extends RelationalTestDB {
type Driver = MemoryDriver
val driver: Driver = new MyMemoryDriver
}

132. Without a Database!
class AccountClientOrderMerchantSupplierTest extends FunSuite with BeforeAndAfter {
val heapDriver = new MyHeapDriver
val profile = heapDriver.driver.profile
import profile.api._
before { initializeEntireSchema() }
test("Create Accounts, Clients, Orders, Merchants, Suppliers and Test") {
db.run(for {
_ <- accounts ++= Seq(Account(...), ...)
_ <- clients ++= Seq(Client(...), ...)
_ <- orders ++= Seq(Order(...), ...)
_ <- merchants ++= Seq(Merchant(...), ...)
_ <- suppliers ++= Seq(Supplier(...), ...)
q <- giantQueryThatCombinesEverything()
_ = { assertRealityIsInLineWithExpectations(accounts, clients, orders, merchants, suppliers) }
} yield ())
}

133. Without a Database!
• Create Schema (~100 `Tables`)
• Initialize Schema with Dozens of Records
• Run Integration Tests with Real Production Data
• 200 ~ 400ms Per Test

134. Testing
Wins!
Obvious Question: Why not write a Memory Driver
for Quill AST, take some notes from slick.memory.QuoterInterpreter ?

135. Ecosystem
v.s.

136. • Event Sourcing (softwaremill/slick-eventsourcing)
• Data Migration (lastland/scala-forklift)
• Cats Integration (RMSone/slick-cats)
• Shapeless Integration (underscoreio/slickless)
• Blocking API (takezoe/blocking-slick)
• Cache (mslinn/quill-gen)
• Annotated Traits (nstojiljkovic/quill-trait)
General

137. • Quill Gen (mslinn/quill-gen)
• “type all the things”-style Generator (olafurpg/scala-db-codegen)
• slick.codegen.SourceCodeGenerator
• Codegen Plugin for SBT (tototoshi/sbt-slick-codegen)
Code Generation

138. • Quill Gen (mslinn/quill-gen)
• Annotated Traits (nstojiljkovic/quill-trait)
• “type all the things”-style Generator (olafurpg/scala-db-codegen)
• slick.codegen.SourceCodeGenerator
• Codegen Plugin for SBT (tototoshi/sbt-slick-codegen)
Code Creation / Generation

139. Ecosystem
Wins!

140. Bonus

141. DataFrame API
orders.as("o")
// Customer has a Location
.join(customers.as("c"), customers("id") === orders("customer"))
.join(destinations.as("d"), destinations("id") === customers("destination"))
// Supplier has a destination
.join(suppliers.as("s"), suppliers("id") === orders("supplier"))
.join(warehouses.as("w"), warehouses("supplier") === suppliers("id"))
.where(warehouses("address") === destinations("address"))
.select(
$"o.timePlaced",
$"c.firstName", $"c.LastName",
$"w.address",
$"d.address"

142. Spark 1.6.x Version
The Spark Dataset API brings the best of RDD and Data
Frames together, for type safety and user functions that run
directly on existing JVM types.
A Dataset is a strongly-typed, immutable collection
of objects that are mapped to a relational schema.

143. Dataset API
orders.as("o")
// Customer has a Location
.join(customers.as("c"), customers("id") === orders("customer"))
.join(destinations.as("d"), destinations("id") === customers("destination"))
// Supplier has a destination
.join(suppliers.as("s"), suppliers("id") === orders("supplier"))
.join(warehouses.as("w"), warehouses("supplier") === suppliers("id"))
.where(warehouses("address") === destinations("address"))
.select(
$"o.timePlaced",
$"c.firstName", $"c.LastName",
$"w.address",
$"d.address"

144. Back to the Trenches?
select
o.timePlaced,
c.firstName, c.LastName,
w.address,
d.address
from
orders o
join customers c on c.id = o.customer
join destinations d on d.id = c.destination
join suppliers s on s.id = o.supplier
join warehouses w on w.supplier = s.id
where
warehouses.address = destinations.address

145. Salvation Cometh... in the form of a QuillSparkContext
def sameAreaOrder = quote {
for {
o <- orders
c <- customers if (c.id === o.customer)
d <- destinations if (d.id === c.destination)
s <- suppliers if (s.id === o.supplier)
w <- warehouses if (w.supplier === s.id)
} yield (
o.timePlaced,
c.firstName, c.LastName,
w.address,
d.address
)
}

146. def sameAreaOrder = quote {
for {
o <- query[Orders]
c <- query[Customers] if (c.id === o.customer)
d <- query[Destinations] if (d.id === c.destination)
s <- query[Suppliers] if (s.id === o.supplier)
w <- query[Warehouses] if (w.supplier === s.id)
} yield (
o.timePlaced,
c.firstName, c.LastName,
w.address,
d.address
)
}
Salvation Cometh... in the form of a QuillSparkContext

147. Let’s Take a Step Back

148. Circa 2015

149. Circa 2015

150. Future of SQL Likely Here

151. Every single one of these already
has a JDBC Driver!

152. SQL is Back!

153. Slick Quill
Usability
Reliability
Extension Friendliness
Streaming
Testing
Ecosystem
Bonus (Spark)

154. Thank You