This document discusses monad transformers in Scala. It begins by introducing the OptionT monad transformer, which lifts an Option into a monad M. It defines the point and map methods for OptionT to make it an instance of the Monad type class. Later sections discuss using monad transformers to compose monads like IO and Option that normally do not compose, and how this allows embedding domain-specific languages within programs.

1. MONAD
TRANSFORMERS
In The Wild

2. speakerdeck.com/u/jrwest/p/monad-transformers

3. TWITTER: @_JRWEST
GITHUB.COM/JRWEST
BLOG.LOOPEDSTRANGE.COM

4. SF SCALA
May 2012
* http://marakana.com/s/scala_typeclassopedia_with_john_kodumal_of_atlassian_video,1198/index.html

5. trait Monad[F[_]] extends Applicative[F] {
def flatMap[A, B](fa: F[A])(f :A=>F[B]):F[B]
}
* monad type class
* flatMap also called bind, >>=

6. def point[A](a: => A): M[A]
def map[A,B](ma: M[A])(f: A => B): M[B]
def flatMap[A,B](ma: M[A])(f: A => M[B]): M[B]
* the functions we care about
* lift pure value, lift pure function, chain “operations”

7. scala> import scalaz.Monad
scala> import scalaz.std.option._
scala> val a = Monad[Option].point(1)
a: Option[Int] = Some(1)
scala> Monad[Option].map(a)(_.toString + "hi")
res2: Option[java.lang.String] = Some(1hi)
scala> Monad[Option].bind(a)(i => if (i < 0) None else Some(i + 1))
res4: Option[Int] = Some(2)
* explicit type class usage in scalaz seven

8. scala> import scalaz.syntax.monad._
import scalaz.syntax.monad._
scala> Option(1).flatMap(i => if (i < 0) None else Some(i+1))
res6: Option[Int] = Some(2)
scala> 1.point[Option].flatMap(...)
res7: Option[Int] = Some(2)
* implicit type class usage in scalaz7 using syntax extensions

9. “A MONADIC FOR
COMPREHENSION IS AN
EMBEDDED PROGRAMMING
LANGUAGE WITH SEMANTICS
DEFINED BY THE MONAD”
* “one intuition of monads” - john

10. MULTIPLE
EFFECTS
Composition

11. Option[A]
* it may not exist

12. SIDE NOTE:
SEMANTICS
* to an extent, you can “choose” the meaning of a monad
* Option -- anon. exceptions -- more narrowly, the exception that something is not there. Validation - monad/not monad - can
mean different things in different contexts

13. IO[Option[A]]
* but side-effects are needed to even look for that value

14. IO[Validation[Throwable,Option[A]]
* and looking for that value may throw exceptions (or fail in some way)

15. IO[(List[String], Validation[Throwable,Option[A])]
* and logging what is going on is necessary

16. MULTIPLE
EFFECTS
A Problem

17. MONADS
DO NOT
COMPOSE
* the problem in theory (core issue)

18. “COMPOSE”?

19. FUNCTORS
DO
COMPOSE
* as well as applicatives

20. trait Functor[F[_]] {
def map[A, B](fa: F[A])(f :A=>B):F[B]
}

21. def composeFunctor[M[_],N[_]](implicit m: Functor[M], n: Functor[N]) =
new Functor[({type MN[A]=[M[N[A]]]})#MN] {
def map[A,B](mna: M[N[A]])(f: A => B): M[N[B]] = ...
}
* generic function that composes any two functors M[_] and N[_]

22. def composeFunctor[M[_],N[_]](implicit m: Functor[M], n: Functor[N]) =
new Functor[({type MN[A]=[M[N[A]]]})#MN] {
def map[A,B](mna: M[N[A]])(f: A => B): M[N[B]] = {
M.map(mna)(na => N.map(na)(f))
}
}

23. scala> Option("abc").map(f)
res1: Option[Int] = Some(3)
scala> List(Option("abc"), Option("d"), Option("ef")).map2(f)
res2: List[Option[Int]] = List(Some(3), Some(1), Some(2))
* can compose functors infinitely deep but...
* scalaz provides method to compose 2, with nice syntatic sugar, easily (map2)

24. def notPossible[M[_],N[_]](implicit m: Monad[M], n: Monad[N]) =
new Monad[({type MN[A]=[M[N[A]]]})#MN] {
def flatMap[A,B](mna: M[N[A]])(f: A => M[N[B]]): M[N[B]] = ...
}
* cannot write the same function for any two monads M[_], N[_]

25. IT !
def notPossible[M[_],N[_]](implicit m: Monad[M], n: Monad[N]) =
Y
new Monad[({type MN[A]=[M[N[A]]]})#MN] {
R
def flatMap[A,B](mna: M[N[A]])(f: A => M[N[B]]): M[N[B]] = ...
}
T
* best way to understand this is attempt to write it yourself
* it won’t compile

26. http://blog.tmorris.net/monads-do-not-compose/
* good resource to dive into this in more detail
* some of previous slides based on above
* provides template, in the form of a gist, for trying this stuff out

27. STAIR
STEPPING
* the problem in practice
*http://www.flickr.com/photos/caliperstudio/2667302181/

28. val a: IO[Option[MyData]] = ...
val b: IO[Option[MyData]] = ...
* have two values that require we communicate w/ outside world to fetch
* those values may not exist (alternative meaning, fetching may result in exceptions that are anonymous)

29. for {
data1 <- a
data2 <- b
} yield {
data1 merge data2 // fail
}
* want to merge the two pieces of data if they both exist

30. for {
// we've escaped IO, fail
d1 <- a.unsafePerformIO
d2 <- b.unsafePerformIO
} yield d1 merge d2
* don’t want to perform the actions until later (don’t escape the IO monad)

31. for {
od1 <- a for {
od2 <- b
od1 <- a
} yield (od1,od2) match {
od2 <- b
case (Some(d1),Some(d2) =>
} yield for {
Option(d1 merge d2)
d1 <- od1
case (a@Some(d1),_)) => a
d2 <- od2
case (_,a@Some(d2)) => a
case _ => None } yield d1 merge d2
}
* may notice the semi-group here
* can also write it w/ an applicative
* this is a contrived example

32. BUT WHAT IF...
def b(data: MyData): IO[Option[MyData]
* even w/ simple example, this minor change throws a monkey wrench in things

33. for {
):
readRes <- readIO(domain)
res <- readRes.fold(
success = _.cata(
some = meta =>
if (meta.enabledStatus /== status) {
writeIO(meta.copy(enabledStatus = status))
} else meta.successNel[BarneyException].pure[IO],
none = new ReadFailure(domain).failNel[AppMetadata].pure[IO]
),
failure = errors => errors.fail[AppMetadata].pure[IO]
)
} yield res
* example of what not to do from something I wrote a while back

34. MULTIPLE
EFFECTS
A Solution

35. case class IOOption[A](run: IO[Option[A]])
define type that boxes box the value, doesn’t need to be a case class, similar to haskell newtype.

36. new Monad[IOOption] {
def point[A](a: => A): IOOption[A] = IOOption(a.point[Option].point[IO])
def map[A,B](fa: IOOption[A])(f: A => B): IOOption[B] =
IOOption(fa.run.map(opt => opt.map(f)))
def flatMap[A, B](fa: IOOption[A])(f :A=>IOOption[B]):IOOption[B] =
IOOption(fa.run.flatMap((o: Option[A]) => o match {
case Some(a) => f(a).run
case None => (None : Option[B]).point[IO]
}))
}
* can define a Monad instance for new type

37. val a: IOOption[MyData] = ...
val b: IOOption[MyData] = ...
val c: IOOption[MyData] = for {
data1 <- a
data2 <- b
} yield {
data1 merge data2
}
val d: IO[Option[MyData]] = c.run
can use new type to improve previous contrived example

38. type MyState[A] = State[StateData,A]
case class MyStateOption[A](run: MyState[Option[A]])
* what if we don’t need effects, but state we can read and write to produce a final optional value and some new state
* State[S,A] where S is fixed is a monad
* can define a new type for that as well

39. new Monad[MyStateOption] { new Monad[IOOption] {
def map[A,B](fa: MyStateOption[A])(f: A => B): MyStateOption[B] = def map[A,B](fa: IOOption[A])(f: A => B): IOOption[B] =
MyStateOption(Functor[MyState].map(fa)(opt => opt.map(f))) IOOption(Functor[IO].map(fa)(opt => opt.map(f)))
def flatMap[A, B](fa: MyStateOption[A])(f :A=>IOOption[B]) = def flatMap[A, B](fa: IOOption[A])(f :A=>IOOption[B]) =
MyStateOption(Monad[MyState]].bind(fa)((o: Option[A]) => o match { IOOption(Monad[IO]].bind(fa)((o: Option[A]) => o match {
case Some(a) => f(a).run case Some(a) => f(a).run
case None => (None : Option[B]).point[MyState] case None => (None : Option[B]).point[IO]
})) }))
} }
* opportunity for more abstraction
* if you were going to do this, not exactly the way you would define these in real code, cheated a bit using {Functor,Monad}.apply

40. case class OptionT[M[_], A](run: M[Option[A]])
define a new type parameterized * -> * and *.

41. case class OptionT[M[_], A](run: M[Option[A]]) {
def map[B](f: A => B)(implicit F: Functor[M]): OptionT[M,B]
def flatMap[B](f: A => OptionT[M,B])(implicit M: Monad[M]): OptionT[M,B]
}
* define map/flatMap a little differently, can be done like previous as typeclass instance but convention is to define the interface
on the transformer and later define typeclass instance using the interface

42. case class OptionT[M[_], A](run: M[Option[A]]) {
def map[B](f: A => B)(implicit F: Functor[M]): OptionT[M,B] =
OptionT[M,B](F.map(run)((o: Option[A]) => o map f))
def flatMap[B](f: A => OptionT[M,B])(implicit M: Monad[M]): OptionT[M,B] =
OptionT[M,B](M.bind(run)((o: Option[A]) => o match {
case Some(a) => f(a).run
case None => M.point((None: Option[B]))
}))
}
* implementations resemble what has already been shown

43. new Monad[IOOption] {
case class OptionT[M[_], A](run: M[Option[A]]) {
def map[A,B](fa: IOOption[A])(f: A => B): IOOption[B] =
def map[B](f: A => B)(implicit F: Functor[M]): OptionT[M,B] =
OptionT[M,B](F.map(run)((o: Option[A]) => o map f)) IOOption(Functor[IO].map(fa)(opt => opt.map(f)))
def flatMap[B](f: A => OptionT[M,B])(implicit M: Monad[M]) = def flatMap[A, B](fa: IOOption[A])(f :A=>IOOption[B]) =
OptionT[M,B](M.bind(run)((o: Option[A]) => o match {
IOOption(Monad[IO]].bind(fa)((o: Option[A]) => o match {
case Some(a) => f(a).run
case Some(a) => f(a).run
case None => M.point((None: Option[B]))
})) case None => (None : Option[B]).point[IO]
} }))
}
* it the generalization of what was written before

44. type FlowState[A] = State[ReqRespData, A]
val f: Option[String] => FlowState[Boolean] = (etag: Option[String]) => {
val a: OptionT[FlowState, Boolean] = for {
// string <- OptionT[FlowState,String]
e <- optionT[FlowState](etag.point[FlowState])
// wrap FlowState[Option[String]] in OptionT
matches <- optionT[FlowState]((requestHeadersL member IfMatch))
} yield matches.split(",").map(_.trim).toList.contains(e)
a getOrElse false // FlowState[Boolean]
}
* check existence of etag in an http request, data lives in state
* has minor bug, doesn’t deal w/ double quotes as written
* https://github.com/stackmob/scalamachine/blob/master/core/src/main/scala/scalamachine/core/v3/
WebmachineDecisions.scala#L282-285

45. val reqCType: OptionT[FlowState,ContentType] = for {
contentType <- optionT[FlowState](
(requestHeadersL member ContentTypeHeader)
)
mediaInfo <- optionT[FlowState](
parseMediaTypes(contentType).headOption.point[FlowState]
)
} yield mediaInfo.mediaRange
* determine content type of the request, data lives in state, may not be specified
* https://github.com/stackmob/scalamachine/blob/master/core/src/main/scala/scalamachine/core/v3/
WebmachineDecisions.scala#L772-775

46. scala> type EitherTString[M[_],A] = EitherT[M,String,A]
defined type alias EitherTString
scala> val items = eitherT[List,String,Int](List(1,2,3,4,5,6).map(Right(_)))
items: scalaz.EitherT[List,String,Int] = ...
* adding features to a “embedded language”

47. for { i <- items } yield print(i)
// 123456
for {
i <- items
_ <- if (i > 4) leftT[List,String,Unit]("fail")
else rightT[List,String,Unit](())
} yield print(i)
// 1234
* adding error handling, and early termination to non-deterministic computation

48. MONAD
TRANSFORMERS
In General

49. MyMonad[A]

50. NAMING CONVENTION
MyMonadT[M[_], A]
* transformer name ends in T

51. BOXES A VALUE
run: M[MyMonad[A]
* value is typically called “run” in scalaz7
* often called “value” in scalaz6 (because of NewType)

52. A MONAD
TRANSFORMER
IS A
MONAD TOO
* i mean, its thats kinda the point of this whole exercise isn’t it :)

53. def optTMonad[M[_] : Monad] = new Monad[({type O[X]=OptionT[M,X]]})#O) {
def point[A](a: => A): OptionT[M,A] = OptionT(a.point[Option].point[M])
def map[A,B](fa: OptionT[M,A])(f: A => B): OptionT[M,B] = fa map f
def flatMap[A, B](fa: OptionT[M,A])(f :A=> OptionT[M,B]): OptionT[M, B] =
fa flatMap f
}
* monad instance definition for OptionT

54. HAS INTERFACE
RESEMBLING UNDERLYING
MONAD’S INTERFACE
* can interact with the monad transformer in a manner similar to working with the actual monad
* same methods, slightly different type signatures
* different from haskell, “feature” of scala, since we can define methods on a type

55. case class OptionT[M[_], A](run: M[Option[A]]) {
def getOrElse[AA >: A](d: => AA)(implicit F: Functor[M]): M[AA] =
F.map(run)((_: Option[A]) getOrElse default)
def orElse[AA >: A](o: OptionT[M,AA])(implicit M: Monad[M]): OptionT[M,AA] =
OptionT[M,AA](M.bind(run) {
case x@Some(_) => M.point(x)
case None => o.run
}
}

56. MONAD
TRANSFORMERS
Stacked Effects

57. TRANSFORMER IS A MONAD
TRANSFORMER CAN WRAP
ANOTHER TRANSFORMER
* at the start, the goal was to stack effects (not just stack 2 effects)
* this makes it possible

58. type VIO[A] = ValidationT[IO,Throwable,A]
def doWork(): VIO[Option[Int]] = ...
val r: OptionT[VIO,Int] = optionT[VIO](doWork())
* wrap the ValidationT with success type Option[A] in an OptionT
* define type alias for connivence -- avoids nasty type lambda syntax inline

59. val action: OptionT[VIO, Boolean] = for {
devDomain <- optionT[VIO] {
validationT(
bucket.fetch[CName]("%s.%s".format(devPrefix,hostname))
).mapFailure(CNameServiceException(_))
}
_ <- optionT[VIO] {
validationT(deleteDomains(devDomain)).map(_.point[Option])
}
} yield true
* code (slightly modified) from one of stackmob’s internal services
* uses Scaliak to fetch hostname data from riak and then remove them
* possible to clean this code up a bit, will discuss shortly (monadtrans)

60. KEEP ON
STACKIN’
ON
* don’t have to stop at 2 levels deep, our new stack is monad too
* each monad/transformer we add to the stack compose more types of effects

61. “ORDER”
MATTERS
* how stack is built, which transformers wrap which monads, determines the overall semantics of the entire stack
* changing that order can, and usually does, change semantics

62. OptionT[FlowState, A]
vs.
StateT[Option,ReqRespData,A]
* what is the difference in semantics between the two?
* type FlowState[A] = State[ReqRespData,A]

63. FlowState[Option[A]]
vs.
Option[State[ReqRespData,A]
* unboxing makes things easier to see
* a state action that returns an optional value vs a state action that may not exist
* the latter probably doesn’t make as much sense in the majority of cases

64. MONADTRANS
The Type Class
* type classes beget more type classes

65. REMOVING REPETITION
===
MORE ABSTRACTION
* previous examples have had a repetitive, annoying, & verbose task
* can be abstracted away...by a type class of course

66. optionT[VIO](validationT(deleteDomains(devDomain)).map(_.point[Option]))
eitherT[List,String,Int](List(1,2,3,4,5,6).map(Right(_)))
resT[FlowState](encodeBodyIfSet(resource).map(_.point[Res]))
* some cases require lifting the value into the monad and then wrap it in the transformer
* from previous examples

67. M[A] -> M[N[A]] -> NT[M[N[_]], A]
* this is basically what we are doing every time
* taking some monad M[A], lifting A into N, a monad we have a transformer for, and then wrapping all of that in N’s monad
transformer

68. trait MonadTrans[F[_[_], _]] {
def liftM[G[_] : Monad, A](a: G[A]): F[G, A]
}
* liftM will do this for any transformer F[_[_],_] and any monad G[_] provided an instance of it is defined for F[_[_],_]

69. def liftM[G[_], A](a: G[A])(implicit G: Monad[G]): OptionT[G, A] =
OptionT[G, A](G.map[A, Option[A]](a)((a: A) => a.point[Option]))
* full definition requires some type ceremony
* https://github.com/scalaz/scalaz/blob/scalaz-seven/core/src/main/scala/scalaz/OptionT.scala#L155-156

70. def liftM[G[_], A](ga: G[A])(implicit G: Monad[G]): ResT[G,A] =
ResT[G,A](G.map(ga)(_.point[Res]))
* implementation for scalamachine’s Res monad
* https://github.com/stackmob/scalamachine/blob/master/scalaz7/src/main/scala/scalamachine/scalaz/res/
ResT.scala#L75-76

71. encodeBodyIfSet(resource).liftM[OptionT]
List(1,2,3).liftM[EitherTString]
validationT(deleteDomains(devDomain)).liftM[OptionT]
* cleanup of previous examples
* method-like syntax requires a bit more work: https://github.com/scalaz/scalaz/blob/scalaz-seven/core/src/main/scala/
scalaz/syntax/MonadSyntax.scala#L9

72. for {
media <- (metadataL >=> contentTypeL).map(_ | ContentType("text/plain")).liftM[ResT]
charset <- (metadataL >=> chosenCharsetL).map2(";charset=" + _).getOrElse("")).liftM[ResT]
_ <- (responseHeadersL += (ContentTypeHeader, media.toHeader + charset)).liftM[ResT]
mbHeader <- (requestHeadersL member AcceptEncoding).liftM[ResT]
decision <- mbHeader >| f7.point[ResTFlow] | chooseEncoding(resource, "identity;q=1.0,*;q=0.5")
} yield decision
* https://github.com/stackmob/scalamachine/blob/master/core/src/main/scala/scalamachine/core/v3/
WebmachineDecisions.scala#L199-205

73. MONAD
TRANSFORMERS
In Review

74. STACKING
MONADS
COMPOSES
EFFECTS
* when monads are stacked an embedded language is being built with multiple effects
* this is not the only intuition of monads/transformers

75. CAN NOT
COMPOSE MONADS
GENERICALLY
* cannot write generic function to compose any two monads M[_], N[_] like we can for any two functors

76. MONAD TRANSFORMERS
COMPOSE M[_] : MONAD WITH
ANY N[_] : MONAD
* can’t compose any two, but can compose a given one with any other

77. MONAD TRANSFORMERS
WRAP OTHER
MONAD TRANSFORMERS
* monad transformers are monads
* so they can be the N[_] : Monad that the transformer composes with its underlying monad

78. MONADTRANS
REDUCES
REPETITION
* often need to take a value that is not entirely lifted into a monad transformer stack and do just that

79. STACK MONADS
DON’T
STAIR-STEP
* monad transformers reduce ugly, stair-stepping or nested code and focuses on core task
* focuses on intuition of mutiple effects instead of handling things haphazardly

80. THANK
YOU
* stackmob, markana, john & atlassian, other sponsors, cosmin

81. QUESTIONS?