A large overview of Apache Maven (concepts, community, evolutions ...)

1. Apache MavenMarch 3rd, 2010 Arnaud Héritier eXo Platform Software Factory Manager

2. Arnaud Héritier Software Factory ManagereXo platform In charge of tools and methods Committer since 2004 and member of the Project Management Committee Coauthor of « Apache Maven » published by Pearson (in French) Contact me : http://aheritier.net Twitter : @aheritier Skype : aheritier

3. Agenda Apache Maven

4. Overview Definition History Concepts Conventions POM Reactor and Modules Inheritance Artifact Repository Dependency Version Profiles BuildLifecycle And Plugins Why did we choose Maven for eXo platform ? Maven, the project choice Maven, the corporate choice

5. Maven’s Ecosystem Repository Managers Quality Management Tests Automation Quality Metrics Reports Project Reports Sonar Continuous Integration IDE Eclipse Idea IntelliJ Netbeans

6. Good & Bad Practices K.I.S.S. Project Organization POM Development

7. Usecases Secure your credentials Build a part of your project using reactor options Automate your release process (at least the technical part) Setup a global mirror

9. Definition Apache Maven is a software project management and comprehension tool. Based on the concept of a project object model (POM) Maven can manage a project's build, binaries, reporting and documentation from a central piece of information.

10. History Initiated in 2001 by Jason Van Zyl in Alexandria, an Apache Jakarta project, Moved to Turbine few months after, Became a Top Level Project in 2003. Maven 2.0 released in September 2005 Maven 3.0 … coming soon !!!

11. Concepts Apache Maven

12. Conventions 1 project = 1 artifact (pom, jar, war, ear, …) Standardized project descriptor (POM) build lifecycle directories layout *.java to compile in src/[main|test]/java *.xml, *.properties needed in classpath and to bundle in archive in src/[main|test]/resources target directory for generated stuffs (sources, classes, …) …

13. POM An XML file (pom.xml) Describing Project identification Project version Project description Build settings Dependencies … <?xml version="1.0" encoding="UTF-8"?> <project> <modelVersion>4.0.0</modelVersion> <groupId>net.aheritier.samples</groupId> <artifactId>simple-webapp</artifactId> <version>1.1-SNAPSHOT</version> <packaging>war</packaging> <name>Simple webapp</name> <inceptionYear>2007</inceptionYear> <dependencies> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-struts</artifactId> <version>2.0.2</version> </dependency> ... </dependencies> </project>

14. Reactor Split your project in sub-modules Maven computes the build order from dependencies between sub-modules. Modules have to be defined in the POM No auto-discovery for performance reasons <project> ... <modules> <module>moduleA</module> <module>moduleB</module> <module>moduleC</module> <module>moduleD</module> <module>moduleE</module> <module>moduleF</module> </modules> ... </project>

15. Inheritance Share settings between projects/modules By default the parent project is supposed to be in the parent directory (../) <parent> <groupId>net.aheritier.sample</groupId> <artifactId>my-parent</artifactId> <version>1.0.0-SNAPSHOT<version> </parent>

16. Inheritance Use a technical inheritance to organize sub-modules Use assembly to package batchs Insert README in all artifacts Use clirr to validate backward compatibility

17. Artifact Repository By default : A central repository http://repo1.maven.org/maven2 Several dozen of Gb of OSS libraries A local repository ${user.home}/.m2/repository All artifacts Used by maven and its plugins Used by your projects (dependencies) Produced by your projects

18. Artifact Repository By default Maven downloads artifacts required by the project or itself from central Downloaded artifacts are stored in the local repository Used to store : Project’s binaries Project’s dependencies Maven and plug-ins binaries

19. Dependencies Without Maven With Maven

20. Dependencies Declaratives groupId + artifactId + version (+ classifier) Type (packaging) : jar, war, pom, ear, … Transitives Lib A needs Lib B Lib B needs Lib C Thus Lib A needs Lib C

21. Dependencies Scope Compile (by default) : Required to build and run the application Runtime : not required to build the application but needed at runtime Ex : taglibs Provided : required to build the application but not needed at runtime (provided by the container) Ex : Servlet API, Driver SGBD, … Test : required to build and launch tests but not needed by the application itself to build and run Ex : Junit, TestNG, DbUnit, … System : local library with absolute path Ex : software products

22. Dependencies Define all dependencies you are using and no more ! If you have optional dependencies Perhaps you should have optional modules instead Cleanup your dependencies with mvndependency:analyze Study your dependencies with mvndependency:tree mvndependency:list

23. Versions Project and dependency versions Two different version variants SNAPSHOT version The version number ends with –SNAPSHOT The project is in development Deliveries are changing over the time and are overridden after each build Artifacts are deployed with a timestamp on remote repositories RELEASE version The version number doesn’t end with –SNAPSHOT Binaries won’t change

24. Versions

25. Versions About SNAPSHOT dependencies Maven allows the configuration of an update policy. The update policy defines the recurrence of checks if there is a new SNAPSHOT version available on the remote repository : always daily (by default) interval:X (a given period in minutes) never Must not be used in a released project They can change thus the release also  The release plugin will enforce it 

26. Versions Range From … to … Maven automatically searches for the corresponding version (using the update policy for released artifacts) To use with caution Risk of non reproducibility of the build Risk of sideeffectson projects depending on yours.

27. Versions Use the versions plugin to update all versions of your project and its modules mvnversions:set –DnewVersion=A.B.C-SNAPSHOT

28. Profiles Allow to modify the default behavior of Maven by overriding/adding some settings Use mvnhelp:active-profilesto debug Explicit activation or deactivation mvn <phases or goals> -PprofileId1,-profileId2 -P!profileId3

30. Activation throughMaven settings<settings> ... <activeProfiles> <activeProfile>profile-1</activeProfile> </activeProfiles> ... </settings>

31. Profiles Activation based on environment variables <profiles> <profile> <activation> <property> <name>!skip-enforce</name> </property> </activation> ... </profile> </profiles> <profiles> <profile> <activation> <property> <name>run-its</name> <value>true</value> </property> </activation> ... </profile> </profiles>

32. Profiles OS / Java settings <profiles> <profile> <activation> <jdk>[1.3,1.6)</jdk> </activation> ... </profile> </profiles> <profiles> <profile> <activation> <os> <name>Windows XP</name> <family>Windows</family> <arch>x86</arch> <version>5.1.2600</version> </os> </activation> ... </profile> </profiles>

33. Profiles Activation on present or missing files <profiles> <profile> <activation> <file> <missing>${project.build.directory}/generated-sources/axistools/wsdl2java/</missing> </file> </activation> ... </profile> </profiles>

34. BuildLifecycle And Plugins Plugin based architecture for a great extensibility Standardized lifecycle to build all types of archetypes

35. BuildLifecycle And Plugins

36. Build Lifecycle And Plugins Many plugins Packaging Reporting IDE integration Miscellaneous tools integration Many locations maven.apache.org mojo.codehaus.org code.google.com … The SWF Team provides a set of coherent and tested plugins using our eXo parent POM

37. Maven or not Maven, that is the question ! Apache Maven

38. Maven, the project’s choice Application’s architecture The project has the freedom to divide the application in modules Maven doesn’t limit the evolution of the application architecture Dependencies management Declarative : Maven automatically downloads them and builds the classpath Transitive : We define only what the module needs itself

39. Maven, the project’s choice Centralizes and automates all development facets (build, tests, releases) One thing it cannot do for you : to develop  Builds Tests Packages Deploys Documents Checks and reports about the quality of developments

40. Maven, the corporate’s choice Widely adopted and known Many developers Developments are standardized Decrease of costs Reuse of knowledge Reuse of configuration fragments Reuse of process and code fragments Product qualityimprovement Reports and monitoring

41. ecosystem Apache Maven

42. Maven’s ecosystem Mavenaloneisnothing You can integrate it with many tools A large set of plug-ins is already available You can define your own plug-ins

43. repository managerS Apache Maven

44. Repository Managers Several products Sonatype Nexus (replaced Proximity) JfrogArtifactory Apache Archiva Basic services Search artifacts Browse repositories Proxy external repositories Host internal repositories Security

45. Secure your builds Deploy a repository manager to proxy externals repositories to : Avoid external network outages Avoid external repository unavailabilities To reduce your company’s external network usage To increase the speed of artifact downloads Additional services offered by such servers : Artifacts procurement to filter what is coming from the outside Staging repository to validate your release before deploying it

46. Nexus at eXo for productivity

47. Nexus at eXo for collaboration Deploy 3rd Party Artifacts Collaborate with Internal Repositories Distribute to the community with Public Repositories Distribute to customers with Private Repositories

48. Nexus at eXo for quality Ease the Burden on Central and others remote repositories Gain Predictability and Scalability Control and Audit Dependencies and Releases Stage releases

49. Quality management Apache Maven

50. Tests Automation Use automated tests as often as you can Many tools are available through Maven JUnit, TestNG – unit tests, Selenium, Canoo – web GUI test, Fitnesse, Greenpepper – functional tests, SoapUI – web services tests JMeter – performances tests And many more frameworks are available to reply your needs

51. Quality Metrics Extract quality metrics from your project and monitor them : Code style (CheckStyle) Bad practices or potential bugs (PMD, FindBugs, Clirr) Tests coverage (Cobertura, Emma, Clover) … You can use blocking rules For example, I break the build if the upward compatibility of public APIs is broken You can use reports Reports are available in a web site generated by Maven Or in a quality dashboard like Sonar

52. Dependency Report

53. Sonar, a quality dashboard

54. Sonar, analyze your project

55. Sonar, Continuous Improvement ?

56. Continuous integration Apache Maven

57. Continuous Integration Setup a continuous integration server to : Have a neutral and unmodified environment to run your tests Quickly react when The build fails (compilation failure for example) A test fails A quality metric is bad Continuously improve the quality of your project and your productivity Many products Jenkins, Hudson, Bamboo, TeamCity, Continuum, Cruisecontrol, …

58. Jenkins, how the weather is ?

59. Jenkins: build, test, check

60. IDE Apache Maven

61. Eclipse Integration from maven (eclipse:eclipse) Allow many customizations Support many versions/variants of eclipse Support many usages (ear …) Doesn’t support projects with “pom” packaging Few support from dev team Many bugs in classpath management Asynchronous You have to regenerate and reload project each time you change a POM)

62. Eclipse Integration from eclipse (m2eclipse) Synchronous Nice UI and services to edit POMs Support projects with “pom” packaging Doesn’t support all usages like EAR with WTP Doesn’t support very well a large number of modules Slow down eclipse on large projects because of a lack of support of incremental build in Maven 2.x and its plugins

63. Eclipse (m2eclipse)

64. Eclipse (m2eclipse)

65. Eclipse (m2eclipse)

66. Idea IntelliJ

67. Netbeans

68. Good & bad practices Apache Maven

69. KISS Apache Maven

70. K.I.S.S. Keep It Simple, Stupid Start from scratch Do not copy/paste what you find without understanding Use only what you need It’s not because maven offers many features that you need to use them Filtering Modules Profiles …

71. PROJECT ORGANIZATIONGOOD & BAD Practices Apache Maven

72. Project bad practices Ignore Maven conventions Except if your are migrating from something else and the target has to be to follow them. Except if they are not compatible with your IDE Different versions in sub-modules In that case they are standalone projects. Too many inheritance levels It makes the POMs maintenance more complex Where should I set this plugin parameter ? In which parent ?

73. Project bad practices Have too many modules Is there a good reason ? Technical constraint ? Team organization ? It increases the build time Many more artifacts to generate Dependencies resolution more complex It involves more complex developments More modules to import in your IDE More modules to update …

74. Project good practices Use the default inheritance : The reactor project is also the parent of its modules. Configuration is easier : No need to redefine SCM settings, site distribution settings …

75. POM GOOD & BAD Practices Apache Maven

76. POM bad practices Dependencies : DON’T confuse dependencies and dependencyManagement Plugins : DON’T confuse plugins and pluginManagement DON’T use AntRunplugin everywhere DON’T let Maven choose plugins versions for you

77. POM bad practices Profiles : DON’T create environment dependant builds DON’T rely on dependencies coming from profiles (there is no transitive activation of profiles) Reporting and quality DON’T activate on an existing project all reports with default configuration DON’T control formatting rules without giving settings for IDEs. DON’T put everything you find in your POM.

78. POM good practices Set versions of dependencies in project parent’s dependencyManagement Set dependencies (groupId, artifactId, scope) in each module they are used Use the dependency plugin (from apache) and versions plugin (from mojo) to analyze, cleanup and update your dependencies.

79. Development good & bad practices Apache Maven

80. Development bad practices DON’T spend your time in the terminal, DON’T exchange libraries through emails, DON’T always use "-Dmaven.test.skip=true” DON’T manually do releases

81. Development good practices Keep up-to-date your version of Maven For example in 2.1 the time of dependencies/modules resolution decreased a lot (Initialization of a project of 150 modules passed from 8 minutes to less than 1) Use the reactor plugin (Maven < 2.1) or native reactor command line options (Maven >= 2.1) to rebuild only a subpart of your project : All modules depending on module XXX All modules used to build XXX Try to not use Maven features not supported by your IDE (resources filtering with the plugineclipse:eclipse)

82. Usecases Apache Maven

83. Secure your credentials Apache Maven

85. arnaud@leopard:~$ mvn --encrypt-master-password toto{dZPuZ74YTJ0HnWHGm4zgfDlruYQNda1xib9vAVf2vvY=}

86. We save the private key in ~/.m2/settings-security.xml<settingssecurity> <master>{dZPuZ74YTJ0HnWHGm4zgfDlruYQNda1xib9vAVf2vvY=}</master> </settingssecurity>

88. <settingsSecurity> <relocation>/Volumes/ArnaudUsbKey/secure/settings-security.xml</relocation></settingsSecurity>

89. You create an encrypted version of your server password

90. arnaud@mbp-arnaud:~$ mvn--encrypt-password titi{SbC9Fl2jA4oHZtz5Fcefp2q1tMXEtBkz9QiKljPiHss=}

91. You register it in your settings

93. Using Reactor Options Options added in maven 2.1 Available in 2.0.x with the maven-reactor-plugin But syntax is longer Allow to control what you want to build in your project

96. Builds only modules B and E

97. Following dependencies order

99. Builds module D (-pl) and all modules it uses as dependencies

100. -am --also-make: If a project list is specified, also make projects that the list depends on

102. Builds module D (-pl) and all modules which depend on it

103. -amd --also-make-dependents: If a project list is specified, also make projects that depend on projects on the list

105. Restarts all the build from module D (-rf)

106. -rf,--resume-from <arg> : Resume reactor from specified project

108. Release of a webapp in 2002 Limited usage of eclipse No WTP (Only some features in WSAD), No ability to produce WARs

109. Release of a webapp in 2002 Many manual tasks Modify settings files Package JARs Copy libraries (external and internal) in a « lib » directory Package WAR (often with a zip command) Tag the code (CVS) Send the package on the integration server using FTP Deploy the package with AS console

110. Release of a webapp in 2002 One problem : The are always problems Error in config files Missing dependencies Missing file Last minute fix which created a bug And many other possibilies .. How long did it take ? When everything is ok : 15 minutes When there’s a problem : ½ day or more

111. Maven Release Plugin Automates the release process from tagging sources to binaries delivery Release plugin main goals: Prepare : To update maven versions and information in POMs and tag the code Perform : To deploy binaries in a maven repository After that you can just automate the deployment on the AS using cargo for example.

112. Maven Release Plugin

113. Configuration and Prerequisites Project version (must be a SNAPSHOT version) Dependencies and plugins versions mustn’t be SNAPSHOTs

114. Troubleshooting Releases Common errorsduring release: Buildwith release profile wastestedbefore and fails Local modifications Current version is not a SNAPSHOT SNAPSHOTs in dependencies and/or plugins Missingsome configuration (scm, distribMgt, …) Tag alreadyexists Unable to deployproject to the Repository Connectionproblems

115. SCM configuration SCM binaries have to be in the PATH SCM credentials have to already be stored or you have to pass them in command line with :–Dusername=XXX –Dpassword=XXX <scm> <connection>scm:svn:http://svn.acme.com/myproject/trunk</connection> <developerConnection>scm:svn:https://svn.acme.com/myproject/trunk</developerConnection> <url>http://fisheye.acme.com/browse/myproject/trunk</url> </scm>

116. Distribution Management Whereyouwant to uploadreleasedbinaries The url of a repositorydedicated for yourproject/corporatemavendeliveries in yourrepository manager <project> <distributionManagement> <repository> <id>repository.acme.com</id> <url>${acme.releases.repo.url}</url> </repository> . . . </distributionManagement> . . . <properties> <acme.releases.repo.url>http://repo.acme.com/acme-releases</acme.releases.repo.url> . . . </properties> </project> This id willbeused in user’smaven settings (~/.m2/settings.xml) Oftenuseful to have a property to test the release process on a fakerepository, to validate a repo manager ...

117. Repository credentials One server entry isrequired per differentrepository id in distribution management of projects In a corporateenvironment, use a unique id for all repositorieshosted on repository managers usingsamecredentials (corporate LDAP …) <settings> ... <servers> <server> <id>repository.acme.com</id> <username>aheritier</username> <password>{ABCDEFGHIJKLMNOPQRSTUVWYZ}</password> </server> ... </servers> ... </settings> This id is the one defined in distributionManagement entry of the project to release

118. Default Release Profile in Super POM This profile isusedwhenyougeneratebinaries of the release with “mvnrelease:perform” By default, generates sources and javadocs jars for each module. <profile> <id>release-profile</id> <activation> <property> <name>performRelease</name> <value>true</value> </property> </activation> <build> <plugins> ... </plugins> </build> </profile> This activation couldbeused in profiles youwant to activate in the release process Configuration to generate sources and javadoc jars with basic setting

119. Custom release profile <project> ... <build> <pluginManagement> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-release-plugin</artifactId> <version>2.0</version> <configuration> <useReleaseProfile>false</useReleaseProfile> <releaseProfiles>myreleaseprofile</releaseProfiles> </configuration> </plugin> </plugins> </pluginManagement> </build> ... <profiles> <profile> <id>myreleaseprofile</id> <build> ... </build> </profile> </profiles> ... </project> Don’t use the default profile Use ourcustomized profile Our customized profile Customize the behavior of the build for a releaseTake care to test isbefore the release !!

120. SETUP a GLObal Mirror Apache Maven

121. Why should we setup a global mirror ? To simplify users and projects settings To control where binaries are coming from To not rely on project’s repositories To use only the corporate repository manager To improve build performances By reducing the number of requests to find a missing artefact

122. How should we setup a global mirror ? <setting> <mirrors> <mirror> <id>global-mirror</id> <mirrorOf>external:*</mirrorOf> <url>http://repo.acme.com/public</url> </mirror> </mirrors> <profiles> <profile> <id>mirror</id> <repositories> <repository> <id>central</id> <url>http://central</url> <releases><enabled>true</enabled></releases> <snapshots><enabled>true</enabled></snapshots> </repository> </repositories> <pluginRepositories> <pluginRepository> <id>central</id> <url>http://central</url> <releases><enabled>true</enabled></releases> <snapshots><enabled>true</enabled></snapshots> </pluginRepository> </pluginRepositories> </profile> </profiles> <activeProfiles> <activeProfile>mirror</activeProfile> </activeProfiles></settings> Send all requests to this url Use « central » id to override default maven configuration Enablesnapshots make the profile active all the time

123. Back to the Future Apache Maven

124. product Apache Maven

125. Apache Maven 2.0.x bugs fix Last release : 2.0.11 No other release of 2.0.x in the future

126. Apache Maven 2.x Evolutions, new features Several important new features in 2.1 like Parallel downloads Encrypted passwords Reactor command line options Last release : 2.2.1

127. Apache Maven 3.x Do not be afraid !!!!! Full compatibility with maven 2.x projects Or at least at 99,99999% 3.0 : October 2010 3.0.1 : November 2010 3.0.2 : January 2011 3.0.3 : March 2011 ~ every 6 weeks

128. Apache Maven 3.x – Why ? To build new foundations for the future The major part of the code was reviewed / rewritten How POMs are constructed How the lifecycle is executed How the plugin manager executes How artifacts are resolved How it can be embedded How dependency injection is done …

129. Apache Maven 3.x - robustness Error & integrity reporting Much improved error reporting where we will provide links to each identifiable problem we know of. There are currently 42 common things that can go wrong. Don't allow builds where versions come from non-project sources like local settings and CLI parameters Don't allow builds where versions come from profiles that have to be activated manually Backward compatibility Several thousands of integration tests

130. Apache Maven 3.x - performances Many optimizations New support of parallel builds of modules New incremental (partial) build To improve IDE integration

131. Apache Maven 3.x – new features Any-source POM If you don’t like XML, choose another DSL Versionless parent elements If you don’t use versions or release plugins to automatically update them Mixins a compositional form of Maven POM configuration Global excludes

132. Apache Maven 3.x What it will change for maven developers ? Lifecycle extension points Plugin extension points Incremental build support Queryable lifecycle Extensible reporting Bye bye Plexus, welcome JSR 330 & Google Guice Welldefined and documented APIs

133. Apache Maven 3.x – New tools mvnsh A cross-platform shell dedicated to maven Tycho Maven ready for OSGI and Eclipse developments

134. In Apache Maven 3.0 ? A backward compatibility near to 100% for projects and plugins A totally new implementation A greater robustness with a better reporting and more readable logs Performances improvements and new parallel builds A better integration for others tools like IDE or continuous integration servers No change in current POM format

135. Community Apache Maven

136. Users Mailing List users@maven.apache.org Trafficstatisticscover a total of 2025 days. Currentsubscribers: 1936 Current digest subscribers: 48 Total posts(2025 days): 89687 Meanposts per day: 44.29 http://pulse.apache.org/ Blue : Number of subscribers Red : Number of messages per day

137. Apache Maven Web Site

138. Dowloads Per month downloads http://people.apache.org/~vgritsenko/stats/projects/maven.html

139. The team 60 committers, More than 30 active in 2009, Several organizations like Sonatype, deliver resources and professional support, A community less isolated : more interactions with Eclipse, Jetty,

140. Commit Statistics

141. Competitors Apache Maven

142. Choose your way of thinking Conventions approach Scripting approach

143. Competitors Ant + Ivy, Easy Ant, Gant, Gradle, Buildr… Script oriented You can do what you want ! Reuse many of Maven conventions (directories layout, …) and services (repositories) but without enforcing them The risk for them : Not being able to evolve due to the too high level of customization proposed to the user. We tried on Maven 1 and it died because of that. It was impossible to create a set of tests to cover all usages. It’s like providing a framework without public API 

144. With scripts oriented builds You can have (if you have good skills) But often you have (moreover after years …)

145. With Maven We dream to deliver (Maven 3.x) But today we have too often (Maven 2.x)

146. Conclusion Apache Maven

147. Conclusion Today, Maven is widely adopted in corporate environments, It provides many services, It has an important and really active community of users and developers Many resources to learn to use it and a professional support are available A product probably far from being perfect but on rails for the future. Maven 3.0 is a new start. Many things to do We need you !

148. Questions ? Apache Maven

149. Licence et copyrights Photos and logos belong to their respective authors/owners Content underCreative Commons 3.0 Attribution — You must attribute the work in the mannerspecified by the author or licensor (but not in anywaythatsuggeststhattheyendorseyou or your use of the work). Noncommercial — You may not use thiswork for commercial purposes. ShareAlike — If you alter, transform, or builduponthiswork, youmaydistribute the resultingworkonlyunder the same or similarlicense to this one. http://creativecommons.org/licenses/by-nc-sa/3.0/us/

150. To go Further … Apache Maven

151. DOCUMENTATIONS Apache Maven

152. Some links The main web site : http://maven.apache.org Project’s team wiki : http://docs.codehaus.org/display/MAVEN Project’suserswiki : http://docs.codehaus.org/display/MAVENUSER

153. Books Nicolas De loofArnaud Héritier Published by Pearson Collection Référence Based on our own experiences with Maven. From beginners to experts. In French only

154. Books Sonatype / O’Reilly : The Definitive Guide http://www.sonatype.com/books Free download Available in several languages

155. Books Apache Maven 2 Effective Implementation Brett Porter, Maria OdeaChing https://www.packtpub.com/apache-maven-2-effective-implementation/book

156. Books Exist Global Better builds with Maven http://www.maestrodev.com/better-build-maven Free download

157. Support Apache Maven

158. Support Mailing lists http://maven.apache.org/mail-lists.html IRC irc.codehaus.org - #maven Forums http://www.developpez.net/ forum maven In French Dedicated support Sonatypeand many others companies