5 Takeaways from Migrating a Library to Scala 3 - Scala Love

@NSilnitsky
@NSilnitsky
5 Takeaways from
migrating a library to Scala 3
Natan Silnitsky Backend Infra TL, Wix.com
natansil.com twitter@NSilnitsky linkedin/natansilnitsky github.com/natansil

@NSilnitsky
Migrating a library to Scala 3
Greyhound
A Scala/Java high-level SDK
for Apache Kafka.
Powered by ZIO
Takeaways from
migrating a library

@NSilnitsky

Upgrade
build tool
Scala
2.13
Scala
3.1
Migrating a library
* code base

SBT
Scala 2.12
Bazel
Scala
2.13
Scala
3.1
Migrating Greyhound library

SBT
Scala 2.12
Bazel
Scala
2.13
Scala
3.1
Migrating Greyhound library
Step 1:
Make SBT Work

Bazel does not support
Scala 3 yet.
And SBT has…
→ a Nice Migration Plugin
→ dedicated syntax
SBT
Scala 2.12
Bazel
Why?
Step 1:
Make SBT Work

Making
SBT Work
1. Change project
directory layout
2. Compile &
Test

@NSilnitsky
@NSilnitsky
lazy val root = project
.in(file("."))
.settings(
name := "greyhound-sbt",
version := "0.1.0-SNAPSHOT",
scalaVersion := "2.12.12",
libraryDependencies ++= Seq(
"com.novocode" % "junit-interface" % "0.11" % "test",
"org.specs2" %% "specs2-junit" % "4.8.3" % "test",
"dev.zio" %% "zio-test-junit" % "1.0.9" % "test",
"org.specs2" %% "specs2-mock" % "4.8.3" % "test",
"dev.zio" %% "zio" % "1.0.9",
"dev.zio" %% "zio-streams" % "1.0.9",
"org.apache.kafka" % "kafka-clients" % "2.4.1",
"org.apache.kafka" %% "kafka" % "2.4.1",
"org.apache.curator" % "curator-test" % "2.12.0",
"com.h2database" % "h2" % "1.4.197"
)
)
build.sbt

SBT
Scala 2.12
Bazel
Scala
2.13
Scala
3.1
Step 1:
Make SBT Work
Step 2:
Switch to Scala 2.13

Why?
Scala
2.13
Scala
3.1
compiler
options &
SBT plugins
interoperable
with 2.13 only
!!

from
SBT
to
Scala 2.13
1. Compile it with
scala Version := "2.13.8"
2. Fix errors in our 3rd-party
library dependencies

@NSilnitsky
@NSilnitsky
.in(file("."))
.settings(
"dev.zio" %% "zio" % "1.0.9",
"com.h2database" % "h2" % "1.4.197"
)
Makes sure that libraries are built with Scala 2.13
build.sbt

@NSilnitsky
@NSilnitsky
.in(file("."))
.settings(
"dev.zio" %% "zio" % "1.0.9",
"com.h2database" % "h2" % "1.4.197"
)
build.sbt
"org.specs2" % "specs2-junit_2.13" % "4.8.3" % "test"

from
SBT
to
Scala 2.13
1. Compile it with
scala Version := "2.13.8"
3. Fix errors in our source code
Scalafix!
2. Fix errors in our 3rd-party
library dependencies
👉 addSbtPlugin("ch.epfl.scala" % "sbt-scalafix" % "0.9.34")

To Your
Codebase
Applies
semantic rules
e.g., migrate to
Scala2.13!
Use Scalafix.

To Your
Codebase
Applies
semantic rules
e.g., migrate to
Scala2.13!
Compiler
Use Scalafix.
Uses a
semantic DB
* model, decouples

@NSilnitsky
@NSilnitsky
[error] found : scala.collection.MapView[com.wixpress.dst.greyhound.core.TopicPartition,Long]
[error] required: Map[com.wixpress.dst.greyhound.core.TopicPartition,com.wixpress.dst.greyhound.core.Offset]

@NSilnitsky
@NSilnitsky
mapValues
2.12 → Map
2.13 → MapView
val offsets = records.groupBy(RecordTopicPartition(_)).mapValues(_.maxBy(_.offset).offset + 1)

@NSilnitsky
@NSilnitsky
mapValues
2.12 → Map
2.13 → MapView
val offsets = records.groupBy(RecordTopicPartition(_)).mapValues(...).toMap

@NSilnitsky
@NSilnitsky
Allow semantic rules
ThisBuild / scalafixDependencies += "org.scala-lang.modules" %% "scala-collection-migrations" % "2.6.0"
.in(file("."))
.settings(
addCompilerPlugin(scalafixSemanticdb),
scalacOptions ++= List("-Yrangepos", "-P:semanticdb:synthetics:on"),
libraryDependencies ++= Seq(...)
)
2.13 Migration rules
build.sbt

@NSilnitsky
@NSilnitsky
> scalafix Collection213Roughly
[info] Running scalafix on 69 Scala sources
[success] Total time: 24 s, completed 16 Feb 2022, 22:22:18
Didn’t work!

@NSilnitsky
@NSilnitsky
val offsets = records.groupBy(RecordTopicPartition(_)).mapValues(...).toMap
> scalafix Collection213Roughly
[info] Running scalafix on 69 Scala sources
[success] Total time: 24 s, completed 16 Feb 2022, 22:22:18
Ended up fixing it manually:

SBT
Scala 2.12
Bazel
Scala
2.13
Scala
3.1
Step 1:
Make SBT Work
Step 2:
Switch to Scala 2.13
Step 3:
Switch to Scala 3

from
Scala 2.13
to
Scala 3 Scala 3
compiler
Scala 2.13
compiler
3rd-party
library dep
Scala 2.13 TASTy Reader can read TASTy format
Scala3 Unpickler
Classpath
Interoperability
can read Pickle format

Just use the migration tool
from
Scala 2.13
to
Scala 3
SBT migration
plugin

@NSilnitsky
helps you migrate your build & code
to Scala3.
Use SBT Migration Plugin.
👉 addSbtPlugin("ch.epfl.scala" % "sbt-scala3-migrate" % "0.5.0")
migrate-libs
migrate-scalacOptions
migrate-syntax
migrate

@NSilnitsky
> migrate-libs root

@NSilnitsky
@NSilnitsky
...
("org.specs2" %% "specs2-junit" % "4.8.3" % "test").cross(CrossVersion.for3Use2_13),
("org.specs2" %% "specs2-mock" % "4.8.3" % "test").cross(CrossVersion.for3Use2_13),
)
)
This crossVersion feature allows to
depend on libraries compiled with 2.13
"org.specs2" % "specs2-mock_2.13" % "4.8.3" % "test"

@NSilnitsky
> migrate-syntax root

@NSilnitsky
> migrate-syntax root
These changes are not mandatory, and caused many new 2.13 compiler
errors → Rolled back
[error] /…/src/main/scala/future-interop/com/wixpress/dst/greyhound/future/GreyhoundRuntime.scala:27:19: type mismatch;
[error] found : zio.clock.Clock.Service.live.type (with underlying type zio.clock.Clock.Service)
[error] required: zio.blocking.Blocking.Service with zio.random.Random.Service with zio.system.System.Service with
zio.console.Console.Service with zio.clock.Clock.Service
[error] Clock.Service.live,
[error] ^
val zenv: Has[Clock.Service] with Has[Console.Service] with Has[System.Service] with Has[Random.Service] with
Has[Blocking.Service]

compiles in scala 3 with options:
> migrate root
-source:3.0-migration
-rewrite

@NSilnitsky
@NSilnitsky
scalacOptions ++= Seq(
"-source:3.0-migration"
),
migration mode:
Compiler forgives on most of the dropped features,
printing warnings in place of errors
Scala compiler options

@NSilnitsky
@NSilnitsky
rewrite mode:
Once your code compiles in the migration mode,
Warnings are resolved by the compiler itself
scalacOptions ++= Seq(
"-source:3.0-migration", “-rewrite”
),
➔ Rewrites are NOT applied if the code compiles in error.
➔ You cannot choose which rules are applied (compiler runs all of them).
Scala compiler options

@NSilnitsky
> migrate root
[warn]
[warn] Note: Unresolved dependencies path:
[error] stack trace is suppressed; run last update for the full output
[error] (update) sbt.librarymanagement.ResolveException: Error downloading org.scalameta:semanticdb-scalac_2.13.8:4.4.31
[error] Not found
[error] Not found
[error] not found: /Users/natans/.ivy2/localorg.scalameta/semanticdb-scalac_2.13.8/4.4.31/ivys/ivy.xml
[error] not found:
https://repo1.maven.org/maven2/org/scalameta/semanticdb-scalac_2.13.8/4.4.31/semanticdb-scalac_2.13.8-4.4.31.pom

@NSilnitsky
> migrate root
[warn]
[warn] Note: Unresolved dependencies path:
[error] stack trace is suppressed; run last update for the full output
[error] (update) sbt.librarymanagement.ResolveException: Error downloading org.scalameta:semanticdb-scalac_2.13.8:4.4.31
[error] Not found
[error] Not found
[error] not found: /Users/natans/.ivy2/localorg.scalameta/semanticdb-scalac_2.13.8/4.4.31/ivys/ivy.xml
[error] not found:
https://repo1.maven.org/maven2/org/scalameta/semanticdb-scalac_2.13.8/4.4.31/semanticdb-scalac_2.13.8-4.4.31.pom
Failed for 2.13.8

@NSilnitsky
> migrate root
Downgraded to 2.13.7

@NSilnitsky
35 | assert(logged)(equalTo(List(
| ^
| Exception occurred while executing macro expansion.
| java.lang.NullPointerException
| at zio.test.Macros$.location(Macros.scala:159)
| at zio.test.Macros$.assert_impl(Macros.scala:175)
"dev.zio" %% "zio-test-junit" % "1.0.9",
"dev.zio" %% "zio-test-junit" % "1.0.13",

@NSilnitsky
> migrate root
Switched to 3.1
> You can now commit the change ❤

@NSilnitsky
5 other build tools for support
➔ Bazel - work in progress - ETA - end of Q1
➔ Gradle - Scala 3 supported version 7.3
➔ Mill - support in 0.9.x
➔ Maven - 4.5.x (scala-maven-plugin)
➔ Native Cross compilation for your library

@NSilnitsky
@NSilnitsky
Takeaways
from
migrating a
library to
Scala3
#1
Use migration tools,
don't do it manually -
it will save you a lot of
time.
* SBT clean

@NSilnitsky
@NSilnitsky
Takeaways
from
migrating a
library to
Scala3
#2
First switch to 2.13.
2.13 is highly compatible -
Unlike the Python 2/3
situation.

@NSilnitsky
@NSilnitsky
Takeaways
from
migrating a
library to
Scala3
#3
Switch syntax lazily.
Take your time with syntax
changes.
“Migrate-syntax” plugin step
would have made migration
much harder

@NSilnitsky
@NSilnitsky
Takeaways
from
migrating a
library to
Scala3
#4
Better to be on SBT,
with other build tools
you’ll have more
manual work.

@NSilnitsky
@NSilnitsky
Takeaways
from
migrating a
library to
Scala3
#5
You can use your
favourite library as
long as it was compiled
with 2.13.
Side note: No macros in Greyhound….

@NSilnitsky
Resources
➔ github.com/natansil/greyhound-sbt
➔ Scalafix installation
➔ 2.13 Migration official documentation
➔ SCALA 2.13’S COLLECTIONS by Julien Richard-Foy
➔ SCALA 3 MIGRATION GUIDE

@NSilnitsky
A Scala ZIO high-level functional SDK for Apache Kafka.
github.com/wix/greyhound
0.2 is out!

@NSilnitsky
@NSilnitsky
Thank
You!
👉 slideshare.net/NatanSilnitsky
Any questions?

@NSilnitsky
@NSilnitsky
Q&A
👉 slideshare.net/NatanSilnitsky
Any questions?

5 Takeaways from Migrating a Library to Scala 3 - Scala Love

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a 5 Takeaways from Migrating a Library to Scala 3 - Scala Love

Semelhante a 5 Takeaways from Migrating a Library to Scala 3 - Scala Love (20)

Mais de Natan Silnitsky

Mais de Natan Silnitsky (20)

Último

Último (20)

5 Takeaways from Migrating a Library to Scala 3 - Scala Love