SlideShare uma empresa Scribd logo
1 de 22
Baixar para ler offline
Managing Binary Compatibility in Scala

                Mirco Dotta

                    Typesafe


            October 13, 2011




            Mirco Dotta   Managing Binary Compatibility in Scala
Introduction Sources of Incompatibility Conclusion


Outline


   Introduction
       Example
       Scala vs. Java

   Sources of Incompatibility
      Type Inferencer
      Type Parameters
      Trait
      Lazy Fields

   Conclusion



                                         Mirco Dotta    Managing Binary Compatibility in Scala
Introduction Sources of Incompatibility Conclusion   Example Scala vs. Java


Example

   Assume Analyzer is part of a library we produce. We decide that
   its API has to evolve as follows:
class Analyzer { // old version                            class Analyzer { // new version
  def analyze(issues: HashMap[ , ]) {...}                    def analyze(issues: Map[ , ]) {...}
}                                                          }

   Further, assume the next expression was compiled against the old
   library
   new Analyzer().analyze(new HashMap[Any,Any])

   Would the compiled code work if run against the new library?

   The answer lies in the bytecode...


                                         Mirco Dotta    Managing Binary Compatibility in Scala
Introduction Sources of Incompatibility Conclusion   Example Scala vs. Java


 Example: Bytecode
      Let’s have a look at the method signature for the two versions:
class Analyzer { // old version                 class Analyzer { // new version
 analyze(Lscala/collection/immutable/HashMap);V analyze(Lscala/collection/immutable/Map);V
}                                               }

      The expression compiled against the old library would look like:
  ...
  invokevirtual #9;// #9 == Analyzer.analyze:(Lscala/collection/immutable/HashMap;)V

      ⇒ The method’s name has been statically resolved at
      compile-time.

      Running it against the new library would result in the JVM
      throwing a NoSuchMethodException.

      ⇒ The evolution of class Analyzer breaks compatibility with
      pre-existing binaries.
                                            Mirco Dotta    Managing Binary Compatibility in Scala
Introduction Sources of Incompatibility Conclusion   Example Scala vs. Java


Is Binary Compatibility a Scala issue?

   The short answer is No. The discussed example can be easily
   ported in Java or other languages targeting the JVM.

   Scala shares with Java many sources of binary incompatibility.

   But Scala offers many language features not available in Java:
          First-class functions.
          Type Inferencer.
          Multiple inheritance via mixin composition (i.e., traits).
          Lazy values.
   . . . Just to cite a few.

   ⇒ Scala code has new “unique” sources of binary incompatibility.


                                         Mirco Dotta    Managing Binary Compatibility in Scala
Introduction Sources of Incompatibility Conclusion      Type Inferencer Type Parameters Trait Lazy Fields


Type Inferencer: Member Signature
     Does the following evolution break binary compatibility?
  class TextAnalyzer { // old version                     class TextAnalyzer { // new version
    def analyze(text: String) = {                           def analyze(text: String) = {
       val issues = Map[String,Any]()                          val issues = new HashMap[String,Any]
       // ...                                                  // ...
       issues                                                  issues
  }}                                                      }}


     Question
     What is the inferred return type of analyze?

     Let’s compare the two methods’ signature.
class TextAnalyzer { // old version                         class TextAnalyzer { // new version
  public scala.collection.immutable.Map                       public scala.collection.immutable.HashMap
     analyze(java.lang.String);                                  analyze(java.lang.String);
}                                                           }


                                           Mirco Dotta       Managing Binary Compatibility in Scala
Introduction Sources of Incompatibility Conclusion   Type Inferencer Type Parameters Trait Lazy Fields


Type Inferencer: Member Signature (2)

   Question
   Can we prevent the method’s signature change?
   That’s easy! The method’s return type has to be explicitly
   declared:
   class TextAnalyzer { // bytecode compatible new version
     def analyze(text: String): Map[String,Any] = {
        val issues = new HashMap[String,Any]
        // ...
        issues
   }}


   Take Home Message
   Always declare the member’s type. If you don’t, you may
   inadvertently change the members’ signature.

                                         Mirco Dotta    Managing Binary Compatibility in Scala
Introduction Sources of Incompatibility Conclusion   Type Inferencer Type Parameters Trait Lazy Fields


Type Parameters & Type Erasure
    Assume NodeImpl <: Node. Does the following evolution break
    binary compatibility?
 class Tree[T <: NodeImpl] { // old version               class Tree[T <: Node] { // new version
   def contains(e: T): Boolean = {...}                      def contains(e: T): Boolean = {...}
 }                                                        }


    Type parameters are erased during compilation, but the erasure of
    Tree.contains is different for the two declarations.
 boolean contains(NodeImpl) {...}                        boolean contains(Node) {...}


    Remember that the JVM looks up methods based on the textual
    representation of the signature, along with the method name.
    Take Home Message
    Type parameters may change the members’ signature and hence
    break binary compatibility.
                                          Mirco Dotta    Managing Binary Compatibility in Scala
Introduction Sources of Incompatibility Conclusion   Type Inferencer Type Parameters Trait Lazy Fields


Trait Compilation

   Traits are a powerful language construct that enables
   multiple-inheritance on top of a runtime – the JVM – that does
   not natively support it.

   Understanding how traits are compiled is crucial if you need to
   ensure release-to-release binary compatibility.

   So, how does the Scala compiler generate the bytecode of a trait?

   There are two key elements:
          A trait is compiled into an interface plus an abstract class
          containing only static methods.
          “Forwarder” methods are injected in classes inheriting traits.

                                         Mirco Dotta    Managing Binary Compatibility in Scala
Introduction Sources of Incompatibility Conclusion    Type Inferencer Type Parameters Trait Lazy Fields


Trait Compilation Explained
     An example will help visualize how traits get compiled:
// declared in a library
trait TypeAnalyzer {
  def analyze(prog: Program) {                            // client code
     // the trait’s method impl code                      class TypingPhase extends TypeAnalyzer
  }
}

     The following is the (pseudo-)bytecode generated by scalac:

interface TypeAnalyzer {                                  class TypingPhase implements TraitAnalyzer {
  void analyze(Program prog);                               // forwarder method injected by scalac
}                                                           void analyze(Program prog) {
abstract class TypeAnalyzer$class {                            // delegates to implementation
  static void analyze(TypeAnalyzer $this,                      TypeAnalyzer$class.analyze(this,prog)
        Program prog) {                                     }
     // the trait’s method impl code                      }
  }
}


                                           Mirco Dotta     Managing Binary Compatibility in Scala
Introduction Sources of Incompatibility Conclusion   Type Inferencer Type Parameters Trait Lazy Fields


Trait: Adding a concrete method




   Question
   Can we add a member in a trait without breaking compatibility
   with pre-existing binaries?




                                         Mirco Dotta    Managing Binary Compatibility in Scala
Introduction Sources of Incompatibility Conclusion    Type Inferencer Type Parameters Trait Lazy Fields


Trait: Adding a concrete method


trait TypeAnalyzer { // new version                   // compiled against the old version
  def analyze(prog: Program) {...}                    class TypingPhase implements TraitAnalyzer {
  def analyze(clazz: ClassInfo) {...}                   // forwarder method injected by scalac
}                                                       void analyze(Program prog) {
                                                           // delegates to implementation
                                                           TypeAnalyzer$class.analyze(this,prog)
//TypeAnalyzer trait compiled                           }
interface TypeAnalyzer {                                // missing concrete implementation!
  void analyze(Program prog);                           ??analyze(ClassInfo clazz)??
  void analyze(ClassInfo clazz);                      }
}
abstract class TypeAnalyzer$class {
  static void analyze(TypeAnalyzer $this,             Take Home Message
        Program prog) {...}
  static void analyze(TypeAnalyzer $this,             Adding a concrete method in a trait
        ClassInfo clazz) {...}                        breaks binary compatibility.
}




                                           Mirco Dotta     Managing Binary Compatibility in Scala
Introduction Sources of Incompatibility Conclusion   Type Inferencer Type Parameters Trait Lazy Fields


Lazy values decompiled

          Lazy values are a Scala language feature that has no
          correspondent in the JVM.
                  ⇒ Their semantic has to be encoded in terms of the available
                  JVM’s primitives.

          A lazy val is encoded into a private field and a getter plus a
          bitmap.
                  The bitmap is used by the getter for ensuring that the private
                  field gets initialized only once.

          The bitmap is of type Int, meaning that a maximum of 32
          lazy value can be correctly handled by it.

          The bitmap is shared with subclasses, so that a new bitmap
          field is created only if strictly necessary.

                                         Mirco Dotta    Managing Binary Compatibility in Scala
Introduction Sources of Incompatibility Conclusion   Type Inferencer Type Parameters Trait Lazy Fields


Lazy to Eager




   Question
   Can we transform a lazy field into an eager one?




                                         Mirco Dotta    Managing Binary Compatibility in Scala
Introduction Sources of Incompatibility Conclusion   Type Inferencer Type Parameters Trait Lazy Fields


Lazy to Eager
   Let’s take a simple example and look at the generated bytecode:
 class ClassAnalyzer { // old version                   class ClassAnalyzer { // new version
   lazy val superclasses: List[Class] = {                 val superclasses: List[Class] = {
      // ...                                                 // ...
   }                                                      }
 }                                                      }

   And here is the bytecode:
 class ClassAnalyzer { // old version
                                                        class ClassAnalyzer { // new version
   private List[Class] superclasses;
                                                          private final List[Class] superclasses;
   volatile int bitmap$0
                                                          public int superclasses();
   public int superclasses();
                                                          // ...
   // ...
                                                        }
 }


   Take Home Message
   Transforming a lazy value into an eager one does preserve
   compatibility with pre-existing binaries.
                                         Mirco Dotta    Managing Binary Compatibility in Scala
Introduction Sources of Incompatibility Conclusion   Type Inferencer Type Parameters Trait Lazy Fields


Eager to Lazy




   Question
   Can we transform an eager field into a lazy one?




                                         Mirco Dotta    Managing Binary Compatibility in Scala
Introduction Sources of Incompatibility Conclusion   Type Inferencer Type Parameters Trait Lazy Fields


Eager to Lazy
   The previous example demonstrates that lazy and eager fields are
   syntactically equivalent, from a client perspective.

   Does that mean that the transformation is safe?

   It depends.
          If you are in a closed world, then the transformation is safe
          (e.g., your class is final).
          Otherwise, you are taking a high risk on breaking the semantic
          of lazy, and no good can come out of that. Remember that
          the bitmap is shared between subclasses.

   Take Home Message
   Transforming an eager value into a lazy one may not preserve
   compatibility with pre-existing binaries.
                                         Mirco Dotta    Managing Binary Compatibility in Scala
Introduction Sources of Incompatibility Conclusion   Conclusion Future Work Scala Migration Manager


Conclusion

          Ensuring release-to-release binary compatibility of Scala
          libraries is possible.
          Though, sometimes it can be difficult to tell if a change in the
          API of a class/trait will break pre-existing binaries.
          In the discussed examples we have seen that:
                  Type inferencer may be at the root of changes in the member’s
                  signature.
                  Type parameters may also modify members’ signature.
                  Traits are a sensible source of binary incompatibilities.
                  Transforming an eager field into a lazy one can break semantic.

   It really looks like library’s maintainers’ life ain’t that easy...

                                         Mirco Dotta    Managing Binary Compatibility in Scala
Introduction Sources of Incompatibility Conclusion   Conclusion Future Work Scala Migration Manager


Scala Migration Manager (MiMa)

   A few months ago we released the Scala Migration Manager!
          It’s free!!
          It can tell you, library maintainers, if your next release is
          binary compatible with the current one.
          It can tell you, libraries users, if two releases of a library are
          binary compatible.
   MiMa can collect and report all sources of “syntactic” binary
   incompatibilities between two releases of a same library.
          “Syntactic” means NO LinkageError (e.g.,
          NoSuchMethodException) will ever be thrown at runtime.

   Now, it’s time for a short demo!

                                         Mirco Dotta    Managing Binary Compatibility in Scala
Introduction Sources of Incompatibility Conclusion   Conclusion Future Work Scala Migration Manager


Future Work


  Reporting binary incompatibilities is only half of the story. We are
  working on a “companion” tool that will help you migrate binary
  incompatibilities.

  For the reporting there are many ideas spinning around. Your
  feedback will help us decide what brings you immediate value

  One that I believe is useful:
          Maven/Sbt integration.




                                         Mirco Dotta    Managing Binary Compatibility in Scala
Introduction Sources of Incompatibility Conclusion      Conclusion Future Work Scala Migration Manager


Scala Migration Manager

   Visit http://typesafe.com/technology/migration-manager
          More information about the Migration Manager
          Download it and try it out, it’s free!

   We want to hear back from you.
          Success stories
          Request new features
          Report bugs
   Want to know more, make sure to get in touch!
                              email: mirco.dotta@typesafe.com, twitter: @mircodotta




                                         Mirco Dotta       Managing Binary Compatibility in Scala
Introduction Sources of Incompatibility Conclusion   Conclusion Future Work Scala Migration Manager


One last thing I didn’t mention...




   We will make the sources public!




                                         Mirco Dotta    Managing Binary Compatibility in Scala

Mais conteúdo relacionado

Mais procurados (20)

Introduction to-programming
Introduction to-programmingIntroduction to-programming
Introduction to-programming
 
Annotations
AnnotationsAnnotations
Annotations
 
java-06inheritance
java-06inheritancejava-06inheritance
java-06inheritance
 
Core java
Core javaCore java
Core java
 
Towards Improving Interface Modularity in Legacy Java Software Through Automa...
Towards Improving Interface Modularity in Legacy Java Software Through Automa...Towards Improving Interface Modularity in Legacy Java Software Through Automa...
Towards Improving Interface Modularity in Legacy Java Software Through Automa...
 
Java tutorial PPT
Java tutorial PPTJava tutorial PPT
Java tutorial PPT
 
Basics of reflection in java
Basics of reflection in javaBasics of reflection in java
Basics of reflection in java
 
Lambda: A Peek Under The Hood - Brian Goetz
Lambda: A Peek Under The Hood - Brian GoetzLambda: A Peek Under The Hood - Brian Goetz
Lambda: A Peek Under The Hood - Brian Goetz
 
Java Presentation For Syntax
Java Presentation For SyntaxJava Presentation For Syntax
Java Presentation For Syntax
 
Java platform
Java platformJava platform
Java platform
 
Reflection
ReflectionReflection
Reflection
 
Core Java introduction | Basics | free course
Core Java introduction | Basics | free course Core Java introduction | Basics | free course
Core Java introduction | Basics | free course
 
Java reflection
Java reflectionJava reflection
Java reflection
 
Ppt chapter03
Ppt chapter03Ppt chapter03
Ppt chapter03
 
Java interfaces
Java interfacesJava interfaces
Java interfaces
 
C#4.0 features
C#4.0 featuresC#4.0 features
C#4.0 features
 
Java OOP Concepts 1st Slide
Java OOP Concepts 1st SlideJava OOP Concepts 1st Slide
Java OOP Concepts 1st Slide
 
Ppt chapter12
Ppt chapter12Ppt chapter12
Ppt chapter12
 
Basics of Java
Basics of JavaBasics of Java
Basics of Java
 
Core java
Core javaCore java
Core java
 

Semelhante a Managing Binary Compatibility in Scala (Scala Lift Off 2011)

Scala Talk at FOSDEM 2009
Scala Talk at FOSDEM 2009Scala Talk at FOSDEM 2009
Scala Talk at FOSDEM 2009Martin Odersky
 
Metaprogramming in Scala 2.10, Eugene Burmako,
Metaprogramming  in Scala 2.10, Eugene Burmako, Metaprogramming  in Scala 2.10, Eugene Burmako,
Metaprogramming in Scala 2.10, Eugene Burmako, Vasil Remeniuk
 
Functional Programming In Jdk8
Functional Programming In Jdk8 Functional Programming In Jdk8
Functional Programming In Jdk8 Bansilal Haudakari
 
scala.reflect, Eugene Burmako
scala.reflect, Eugene Burmakoscala.reflect, Eugene Burmako
scala.reflect, Eugene BurmakoVasil Remeniuk
 
Евгений Бурмако «scala.reflect»
Евгений Бурмако «scala.reflect»Евгений Бурмако «scala.reflect»
Евгений Бурмако «scala.reflect»e-Legion
 
Discover Tasty Query: The library for Scala program analysis
Discover Tasty Query: The library for Scala program analysisDiscover Tasty Query: The library for Scala program analysis
Discover Tasty Query: The library for Scala program analysisJames Thompson
 
Framework engineering JCO 2011
Framework engineering JCO 2011Framework engineering JCO 2011
Framework engineering JCO 2011YoungSu Son
 
Functional programming in java 8 by harmeet singh
Functional programming in java 8 by harmeet singhFunctional programming in java 8 by harmeet singh
Functional programming in java 8 by harmeet singhHarmeet Singh(Taara)
 
Stepping Up : A Brief Intro to Scala
Stepping Up : A Brief Intro to ScalaStepping Up : A Brief Intro to Scala
Stepping Up : A Brief Intro to ScalaDerek Chen-Becker
 
whats new in java 8
whats new in java 8 whats new in java 8
whats new in java 8 Dori Waldman
 
Postdoc symposium - A Logic Meta-Programming Foundation for Example-Driven Pa...
Postdoc symposium - A Logic Meta-Programming Foundation for Example-Driven Pa...Postdoc symposium - A Logic Meta-Programming Foundation for Example-Driven Pa...
Postdoc symposium - A Logic Meta-Programming Foundation for Example-Driven Pa...ICSM 2011
 
A Logic Meta-Programming Foundation for Example-Driven Pattern Detection in O...
A Logic Meta-Programming Foundation for Example-Driven Pattern Detection in O...A Logic Meta-Programming Foundation for Example-Driven Pattern Detection in O...
A Logic Meta-Programming Foundation for Example-Driven Pattern Detection in O...Coen De Roover
 
Lecture2_MCS4_Evening.pptx
Lecture2_MCS4_Evening.pptxLecture2_MCS4_Evening.pptx
Lecture2_MCS4_Evening.pptxSaqlainYaqub1
 
Core Java Tutorials by Mahika Tutorials
Core Java Tutorials by Mahika TutorialsCore Java Tutorials by Mahika Tutorials
Core Java Tutorials by Mahika TutorialsMahika Tutorials
 
JavaParser - A tool to generate, analyze and refactor Java code
JavaParser - A tool to generate, analyze and refactor Java codeJavaParser - A tool to generate, analyze and refactor Java code
JavaParser - A tool to generate, analyze and refactor Java codeFederico Tomassetti
 

Semelhante a Managing Binary Compatibility in Scala (Scala Lift Off 2011) (20)

Scala Talk at FOSDEM 2009
Scala Talk at FOSDEM 2009Scala Talk at FOSDEM 2009
Scala Talk at FOSDEM 2009
 
Scala Days NYC 2016
Scala Days NYC 2016Scala Days NYC 2016
Scala Days NYC 2016
 
JVM
JVMJVM
JVM
 
Metaprogramming in Scala 2.10, Eugene Burmako,
Metaprogramming  in Scala 2.10, Eugene Burmako, Metaprogramming  in Scala 2.10, Eugene Burmako,
Metaprogramming in Scala 2.10, Eugene Burmako,
 
Functional Programming In Jdk8
Functional Programming In Jdk8 Functional Programming In Jdk8
Functional Programming In Jdk8
 
scala.reflect, Eugene Burmako
scala.reflect, Eugene Burmakoscala.reflect, Eugene Burmako
scala.reflect, Eugene Burmako
 
Евгений Бурмако «scala.reflect»
Евгений Бурмако «scala.reflect»Евгений Бурмако «scala.reflect»
Евгений Бурмако «scala.reflect»
 
Discover Tasty Query: The library for Scala program analysis
Discover Tasty Query: The library for Scala program analysisDiscover Tasty Query: The library for Scala program analysis
Discover Tasty Query: The library for Scala program analysis
 
Java training
Java trainingJava training
Java training
 
Framework engineering JCO 2011
Framework engineering JCO 2011Framework engineering JCO 2011
Framework engineering JCO 2011
 
Functional programming in java 8 by harmeet singh
Functional programming in java 8 by harmeet singhFunctional programming in java 8 by harmeet singh
Functional programming in java 8 by harmeet singh
 
Stepping Up : A Brief Intro to Scala
Stepping Up : A Brief Intro to ScalaStepping Up : A Brief Intro to Scala
Stepping Up : A Brief Intro to Scala
 
whats new in java 8
whats new in java 8 whats new in java 8
whats new in java 8
 
Postdoc symposium - A Logic Meta-Programming Foundation for Example-Driven Pa...
Postdoc symposium - A Logic Meta-Programming Foundation for Example-Driven Pa...Postdoc symposium - A Logic Meta-Programming Foundation for Example-Driven Pa...
Postdoc symposium - A Logic Meta-Programming Foundation for Example-Driven Pa...
 
A Logic Meta-Programming Foundation for Example-Driven Pattern Detection in O...
A Logic Meta-Programming Foundation for Example-Driven Pattern Detection in O...A Logic Meta-Programming Foundation for Example-Driven Pattern Detection in O...
A Logic Meta-Programming Foundation for Example-Driven Pattern Detection in O...
 
Java Annotations
Java AnnotationsJava Annotations
Java Annotations
 
Core_Java_Interview.pdf
Core_Java_Interview.pdfCore_Java_Interview.pdf
Core_Java_Interview.pdf
 
Lecture2_MCS4_Evening.pptx
Lecture2_MCS4_Evening.pptxLecture2_MCS4_Evening.pptx
Lecture2_MCS4_Evening.pptx
 
Core Java Tutorials by Mahika Tutorials
Core Java Tutorials by Mahika TutorialsCore Java Tutorials by Mahika Tutorials
Core Java Tutorials by Mahika Tutorials
 
JavaParser - A tool to generate, analyze and refactor Java code
JavaParser - A tool to generate, analyze and refactor Java codeJavaParser - A tool to generate, analyze and refactor Java code
JavaParser - A tool to generate, analyze and refactor Java code
 

Mais de mircodotta

Scala Past, Present & Future
Scala Past, Present & FutureScala Past, Present & Future
Scala Past, Present & Futuremircodotta
 
Lightbend Lagom: Microservices Just Right (Scala Days 2016 Berlin)
Lightbend Lagom: Microservices Just Right (Scala Days 2016 Berlin)Lightbend Lagom: Microservices Just Right (Scala Days 2016 Berlin)
Lightbend Lagom: Microservices Just Right (Scala Days 2016 Berlin)mircodotta
 
Lightbend Lagom: Microservices Just Right
Lightbend Lagom: Microservices Just RightLightbend Lagom: Microservices Just Right
Lightbend Lagom: Microservices Just Rightmircodotta
 
Akka streams scala italy2015
Akka streams scala italy2015Akka streams scala italy2015
Akka streams scala italy2015mircodotta
 
Go Reactive: Event-Driven, Scalable, Resilient & Responsive Systems (Soft-Sha...
Go Reactive: Event-Driven, Scalable, Resilient & Responsive Systems (Soft-Sha...Go Reactive: Event-Driven, Scalable, Resilient & Responsive Systems (Soft-Sha...
Go Reactive: Event-Driven, Scalable, Resilient & Responsive Systems (Soft-Sha...mircodotta
 
Effective Scala (JavaDay Riga 2013)
Effective Scala (JavaDay Riga 2013)Effective Scala (JavaDay Riga 2013)
Effective Scala (JavaDay Riga 2013)mircodotta
 
Effective Scala (SoftShake 2013)
Effective Scala (SoftShake 2013)Effective Scala (SoftShake 2013)
Effective Scala (SoftShake 2013)mircodotta
 
Scala: Simplifying Development
Scala: Simplifying DevelopmentScala: Simplifying Development
Scala: Simplifying Developmentmircodotta
 

Mais de mircodotta (9)

Scala Past, Present & Future
Scala Past, Present & FutureScala Past, Present & Future
Scala Past, Present & Future
 
Lightbend Lagom: Microservices Just Right (Scala Days 2016 Berlin)
Lightbend Lagom: Microservices Just Right (Scala Days 2016 Berlin)Lightbend Lagom: Microservices Just Right (Scala Days 2016 Berlin)
Lightbend Lagom: Microservices Just Right (Scala Days 2016 Berlin)
 
Lightbend Lagom: Microservices Just Right
Lightbend Lagom: Microservices Just RightLightbend Lagom: Microservices Just Right
Lightbend Lagom: Microservices Just Right
 
Akka streams scala italy2015
Akka streams scala italy2015Akka streams scala italy2015
Akka streams scala italy2015
 
Akka streams
Akka streamsAkka streams
Akka streams
 
Go Reactive: Event-Driven, Scalable, Resilient & Responsive Systems (Soft-Sha...
Go Reactive: Event-Driven, Scalable, Resilient & Responsive Systems (Soft-Sha...Go Reactive: Event-Driven, Scalable, Resilient & Responsive Systems (Soft-Sha...
Go Reactive: Event-Driven, Scalable, Resilient & Responsive Systems (Soft-Sha...
 
Effective Scala (JavaDay Riga 2013)
Effective Scala (JavaDay Riga 2013)Effective Scala (JavaDay Riga 2013)
Effective Scala (JavaDay Riga 2013)
 
Effective Scala (SoftShake 2013)
Effective Scala (SoftShake 2013)Effective Scala (SoftShake 2013)
Effective Scala (SoftShake 2013)
 
Scala: Simplifying Development
Scala: Simplifying DevelopmentScala: Simplifying Development
Scala: Simplifying Development
 

Último

Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 

Último (20)

Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 

Managing Binary Compatibility in Scala (Scala Lift Off 2011)

  • 1. Managing Binary Compatibility in Scala Mirco Dotta Typesafe October 13, 2011 Mirco Dotta Managing Binary Compatibility in Scala
  • 2. Introduction Sources of Incompatibility Conclusion Outline Introduction Example Scala vs. Java Sources of Incompatibility Type Inferencer Type Parameters Trait Lazy Fields Conclusion Mirco Dotta Managing Binary Compatibility in Scala
  • 3. Introduction Sources of Incompatibility Conclusion Example Scala vs. Java Example Assume Analyzer is part of a library we produce. We decide that its API has to evolve as follows: class Analyzer { // old version class Analyzer { // new version def analyze(issues: HashMap[ , ]) {...} def analyze(issues: Map[ , ]) {...} } } Further, assume the next expression was compiled against the old library new Analyzer().analyze(new HashMap[Any,Any]) Would the compiled code work if run against the new library? The answer lies in the bytecode... Mirco Dotta Managing Binary Compatibility in Scala
  • 4. Introduction Sources of Incompatibility Conclusion Example Scala vs. Java Example: Bytecode Let’s have a look at the method signature for the two versions: class Analyzer { // old version class Analyzer { // new version analyze(Lscala/collection/immutable/HashMap);V analyze(Lscala/collection/immutable/Map);V } } The expression compiled against the old library would look like: ... invokevirtual #9;// #9 == Analyzer.analyze:(Lscala/collection/immutable/HashMap;)V ⇒ The method’s name has been statically resolved at compile-time. Running it against the new library would result in the JVM throwing a NoSuchMethodException. ⇒ The evolution of class Analyzer breaks compatibility with pre-existing binaries. Mirco Dotta Managing Binary Compatibility in Scala
  • 5. Introduction Sources of Incompatibility Conclusion Example Scala vs. Java Is Binary Compatibility a Scala issue? The short answer is No. The discussed example can be easily ported in Java or other languages targeting the JVM. Scala shares with Java many sources of binary incompatibility. But Scala offers many language features not available in Java: First-class functions. Type Inferencer. Multiple inheritance via mixin composition (i.e., traits). Lazy values. . . . Just to cite a few. ⇒ Scala code has new “unique” sources of binary incompatibility. Mirco Dotta Managing Binary Compatibility in Scala
  • 6. Introduction Sources of Incompatibility Conclusion Type Inferencer Type Parameters Trait Lazy Fields Type Inferencer: Member Signature Does the following evolution break binary compatibility? class TextAnalyzer { // old version class TextAnalyzer { // new version def analyze(text: String) = { def analyze(text: String) = { val issues = Map[String,Any]() val issues = new HashMap[String,Any] // ... // ... issues issues }} }} Question What is the inferred return type of analyze? Let’s compare the two methods’ signature. class TextAnalyzer { // old version class TextAnalyzer { // new version public scala.collection.immutable.Map public scala.collection.immutable.HashMap analyze(java.lang.String); analyze(java.lang.String); } } Mirco Dotta Managing Binary Compatibility in Scala
  • 7. Introduction Sources of Incompatibility Conclusion Type Inferencer Type Parameters Trait Lazy Fields Type Inferencer: Member Signature (2) Question Can we prevent the method’s signature change? That’s easy! The method’s return type has to be explicitly declared: class TextAnalyzer { // bytecode compatible new version def analyze(text: String): Map[String,Any] = { val issues = new HashMap[String,Any] // ... issues }} Take Home Message Always declare the member’s type. If you don’t, you may inadvertently change the members’ signature. Mirco Dotta Managing Binary Compatibility in Scala
  • 8. Introduction Sources of Incompatibility Conclusion Type Inferencer Type Parameters Trait Lazy Fields Type Parameters & Type Erasure Assume NodeImpl <: Node. Does the following evolution break binary compatibility? class Tree[T <: NodeImpl] { // old version class Tree[T <: Node] { // new version def contains(e: T): Boolean = {...} def contains(e: T): Boolean = {...} } } Type parameters are erased during compilation, but the erasure of Tree.contains is different for the two declarations. boolean contains(NodeImpl) {...} boolean contains(Node) {...} Remember that the JVM looks up methods based on the textual representation of the signature, along with the method name. Take Home Message Type parameters may change the members’ signature and hence break binary compatibility. Mirco Dotta Managing Binary Compatibility in Scala
  • 9. Introduction Sources of Incompatibility Conclusion Type Inferencer Type Parameters Trait Lazy Fields Trait Compilation Traits are a powerful language construct that enables multiple-inheritance on top of a runtime – the JVM – that does not natively support it. Understanding how traits are compiled is crucial if you need to ensure release-to-release binary compatibility. So, how does the Scala compiler generate the bytecode of a trait? There are two key elements: A trait is compiled into an interface plus an abstract class containing only static methods. “Forwarder” methods are injected in classes inheriting traits. Mirco Dotta Managing Binary Compatibility in Scala
  • 10. Introduction Sources of Incompatibility Conclusion Type Inferencer Type Parameters Trait Lazy Fields Trait Compilation Explained An example will help visualize how traits get compiled: // declared in a library trait TypeAnalyzer { def analyze(prog: Program) { // client code // the trait’s method impl code class TypingPhase extends TypeAnalyzer } } The following is the (pseudo-)bytecode generated by scalac: interface TypeAnalyzer { class TypingPhase implements TraitAnalyzer { void analyze(Program prog); // forwarder method injected by scalac } void analyze(Program prog) { abstract class TypeAnalyzer$class { // delegates to implementation static void analyze(TypeAnalyzer $this, TypeAnalyzer$class.analyze(this,prog) Program prog) { } // the trait’s method impl code } } } Mirco Dotta Managing Binary Compatibility in Scala
  • 11. Introduction Sources of Incompatibility Conclusion Type Inferencer Type Parameters Trait Lazy Fields Trait: Adding a concrete method Question Can we add a member in a trait without breaking compatibility with pre-existing binaries? Mirco Dotta Managing Binary Compatibility in Scala
  • 12. Introduction Sources of Incompatibility Conclusion Type Inferencer Type Parameters Trait Lazy Fields Trait: Adding a concrete method trait TypeAnalyzer { // new version // compiled against the old version def analyze(prog: Program) {...} class TypingPhase implements TraitAnalyzer { def analyze(clazz: ClassInfo) {...} // forwarder method injected by scalac } void analyze(Program prog) { // delegates to implementation TypeAnalyzer$class.analyze(this,prog) //TypeAnalyzer trait compiled } interface TypeAnalyzer { // missing concrete implementation! void analyze(Program prog); ??analyze(ClassInfo clazz)?? void analyze(ClassInfo clazz); } } abstract class TypeAnalyzer$class { static void analyze(TypeAnalyzer $this, Take Home Message Program prog) {...} static void analyze(TypeAnalyzer $this, Adding a concrete method in a trait ClassInfo clazz) {...} breaks binary compatibility. } Mirco Dotta Managing Binary Compatibility in Scala
  • 13. Introduction Sources of Incompatibility Conclusion Type Inferencer Type Parameters Trait Lazy Fields Lazy values decompiled Lazy values are a Scala language feature that has no correspondent in the JVM. ⇒ Their semantic has to be encoded in terms of the available JVM’s primitives. A lazy val is encoded into a private field and a getter plus a bitmap. The bitmap is used by the getter for ensuring that the private field gets initialized only once. The bitmap is of type Int, meaning that a maximum of 32 lazy value can be correctly handled by it. The bitmap is shared with subclasses, so that a new bitmap field is created only if strictly necessary. Mirco Dotta Managing Binary Compatibility in Scala
  • 14. Introduction Sources of Incompatibility Conclusion Type Inferencer Type Parameters Trait Lazy Fields Lazy to Eager Question Can we transform a lazy field into an eager one? Mirco Dotta Managing Binary Compatibility in Scala
  • 15. Introduction Sources of Incompatibility Conclusion Type Inferencer Type Parameters Trait Lazy Fields Lazy to Eager Let’s take a simple example and look at the generated bytecode: class ClassAnalyzer { // old version class ClassAnalyzer { // new version lazy val superclasses: List[Class] = { val superclasses: List[Class] = { // ... // ... } } } } And here is the bytecode: class ClassAnalyzer { // old version class ClassAnalyzer { // new version private List[Class] superclasses; private final List[Class] superclasses; volatile int bitmap$0 public int superclasses(); public int superclasses(); // ... // ... } } Take Home Message Transforming a lazy value into an eager one does preserve compatibility with pre-existing binaries. Mirco Dotta Managing Binary Compatibility in Scala
  • 16. Introduction Sources of Incompatibility Conclusion Type Inferencer Type Parameters Trait Lazy Fields Eager to Lazy Question Can we transform an eager field into a lazy one? Mirco Dotta Managing Binary Compatibility in Scala
  • 17. Introduction Sources of Incompatibility Conclusion Type Inferencer Type Parameters Trait Lazy Fields Eager to Lazy The previous example demonstrates that lazy and eager fields are syntactically equivalent, from a client perspective. Does that mean that the transformation is safe? It depends. If you are in a closed world, then the transformation is safe (e.g., your class is final). Otherwise, you are taking a high risk on breaking the semantic of lazy, and no good can come out of that. Remember that the bitmap is shared between subclasses. Take Home Message Transforming an eager value into a lazy one may not preserve compatibility with pre-existing binaries. Mirco Dotta Managing Binary Compatibility in Scala
  • 18. Introduction Sources of Incompatibility Conclusion Conclusion Future Work Scala Migration Manager Conclusion Ensuring release-to-release binary compatibility of Scala libraries is possible. Though, sometimes it can be difficult to tell if a change in the API of a class/trait will break pre-existing binaries. In the discussed examples we have seen that: Type inferencer may be at the root of changes in the member’s signature. Type parameters may also modify members’ signature. Traits are a sensible source of binary incompatibilities. Transforming an eager field into a lazy one can break semantic. It really looks like library’s maintainers’ life ain’t that easy... Mirco Dotta Managing Binary Compatibility in Scala
  • 19. Introduction Sources of Incompatibility Conclusion Conclusion Future Work Scala Migration Manager Scala Migration Manager (MiMa) A few months ago we released the Scala Migration Manager! It’s free!! It can tell you, library maintainers, if your next release is binary compatible with the current one. It can tell you, libraries users, if two releases of a library are binary compatible. MiMa can collect and report all sources of “syntactic” binary incompatibilities between two releases of a same library. “Syntactic” means NO LinkageError (e.g., NoSuchMethodException) will ever be thrown at runtime. Now, it’s time for a short demo! Mirco Dotta Managing Binary Compatibility in Scala
  • 20. Introduction Sources of Incompatibility Conclusion Conclusion Future Work Scala Migration Manager Future Work Reporting binary incompatibilities is only half of the story. We are working on a “companion” tool that will help you migrate binary incompatibilities. For the reporting there are many ideas spinning around. Your feedback will help us decide what brings you immediate value One that I believe is useful: Maven/Sbt integration. Mirco Dotta Managing Binary Compatibility in Scala
  • 21. Introduction Sources of Incompatibility Conclusion Conclusion Future Work Scala Migration Manager Scala Migration Manager Visit http://typesafe.com/technology/migration-manager More information about the Migration Manager Download it and try it out, it’s free! We want to hear back from you. Success stories Request new features Report bugs Want to know more, make sure to get in touch! email: mirco.dotta@typesafe.com, twitter: @mircodotta Mirco Dotta Managing Binary Compatibility in Scala
  • 22. Introduction Sources of Incompatibility Conclusion Conclusion Future Work Scala Migration Manager One last thing I didn’t mention... We will make the sources public! Mirco Dotta Managing Binary Compatibility in Scala