Gr8conf EU 2013 Speed up your development: GroovyServ and Grails Improx Plugin
•
10 likes•4,321 views
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Gr8conf EU 2013 Speed up your development: GroovyServ and Grails Improx Plugin
Similar to Gr8conf EU 2013 Speed up your development: GroovyServ and Grails Improx Plugin (20)
More from Yasuharu Nakano
Recently uploaded
Recently uploaded (20)
Gr8conf EU 2013 Speed up your development: GroovyServ and Grails Improx Plugin
- 1. Speed Up Your Development: GroovyServ & Grails Improx Plugin @nobeans Yasuharu NAKANO
- 2. GroovyServ
- 3. In case of Developing a Groovy Script
- 4. Write Run Common Strategy: “Trial and Error”
- 5. $ time groovy -‐e “println ‘Hello, Groovy’” Hello, Groovy real 0m0.823s user 0m1.016s sys 0m0.096s But, It takes 1 sec!
- 6. N times Trial and Error takes N sec.
- 7. “Many a little makes a mickle”
- 8. Waiting time makes me Forget next action
- 9. Why so slow? • JVM initialization • Loading Many Many Classes • .... from Disk http://en.wikipedia.org/wiki/Java_performance#Startup_time “ It seems that much of the startup time is due to IO- bound operations rather than JVM initialization or class loading (the rt.jar class data file alone is 40 MB and the JVM must seek a lot of data in this huge file).
- 11. Any solutions?
- 12. $ groovyConsole
- 14. groovyConsole • Handy for trying Java/Groovy APIs, etc. • But features as editor aren’t so rich: • No code completion • No syntax checking • No auto import • etc. • I want to use my favorite IDE/editor!
- 15. Other solutions?
- 16. GroovyServ “GroovyServ makes Groovy’s startup time much faster, by pre-invoking Groovy as a server.” http://kobo.github.io/groovyserv/
- 18. $ time groovyclient -‐e “println ‘Hello, GroovyServ’” Hello, GroovyServ real 0m0.031s user 0m0.001s sys 0m0.002s Normal Groovy: real 0m0.823s user 0m1.016s sys 0m0.096s
- 19. Normal Groovy User Groovy Script Shell Environment JavaVM File System C LASSPATH cm d line args stdinstdout stderr Ctrl-C otherEN Vs exitstatus System .in System .out System .err groovy User Groovy Script
- 20. Shell Environment JavaVM File System User Groovy Script C LASSPATH cm d line args stdinstdout stderr Ctrl-C otherEN Vs ≪file≫ Authtoken exitstatus TCP/IP System .in System .out System .err groovyservergroovyclient GroovyServ
- 21. In case of Developing a Groovy Script
- 22. GroovyServ Way • Write code with your favorite IDE/editor • Run the code by using GroovyServ as External Tool • You can also see the result on a console of IDE
- 24. Available Editors/IDEs • IntelliJ IDEA • Eclipse/STS/GGTS • Sublime Text 2 • Vim + QuickRun.vim • and more...
- 25. GroovyServ “GroovyServ makes Groovy’s startup time much faster, by pre-invoking Groovy as a server.” http://kobo.github.io/groovyserv/
- 26. How to Install
- 27. How to Install • Downloadable Binaries for • Mac OS X (built at Mac OS X 10.8 (x86_64)) • Windows • bundled in Groovy Windows Installer • (built at WindowsXP (32bit) (x86)) • (built at Windows7 (64bit) (AMD64)) • Linux (i386) (linked with glibc 2.9) • Self-build (required Gradle) • Homebrew (only Mac) • $ brew install groovyserv • http://kobo.github.io/groovyserv/howtoinstall.html
- 28. No setting required • If there is a binary set, you can use it: • $ /xxx/groovyserv/bin/groovyclient • Adding to PATH, you can easily use it: • $ groovyclient
- 29. How to Use
- 31. groovyclient • Invokes a groovy file or one-liner code, by sending a request to server • Also invokes groovyserver process automatically if not exists
- 32. groovyserver • Invokes a JVM process as a server. • Should be used only when you want to specify special options. • e.g. • -‐v verbose option • -‐-‐allow-‐from for remote feature
- 33. Available for Other JVM Languages http://nobeans-en.blogspot.jp/2011/07/high-speed-start-up- jythonclojure-by.html
- 34. Jython $ alias gythonserver="env CLASSPATH=/your/jython.jar groovyserver" $ alias gython="groovyclient -‐e 'import org.python.util.jython; jython.main(args)' -‐-‐" $ gythonserver -‐r ... $ time gython -‐c "print('Hello')" Hello real 0m0.057s user 0m0.001s sys 0m0.004s
- 35. Clojure $ alias glojureserver="env CLASSPATH=/your/clojure.jar groovyserver" $ alias glojure="groovyclient -‐e 'import clojure.main;main.main(args)' -‐-‐" $ glojureserver -‐r ... $ time glojure -‐e "(println 'Hello)" Hello real 0m0.053s user 0m0.001s sys 0m0.004s main method of closure.main class
- 36. • A script to get schedule data from groupware • As editor’s macro • As mail filter • etc. Using at Runtime
- 37. Other Features • Access Control • Propagation of CLASSPATH • Propagation of Environment Variables • Handling System#exit() • Dynamic CWD
- 39. In case of Developing Grails Application
- 40. Grails’ Test supports is Great.
- 41. How to run tests • All tests • grails test-‐app
- 42. While I’m writing a test, I want to run only it.
- 43. I want to run the test as fast as possible.
- 44. How to run a specified test fast • Run only a unit test • test-‐app unit: <FQCN of TestClass> • Run only a integration test • test-‐app integration: <FQCN of TestClass> • Run only a functional test • test-‐app functional: <FQCN of TestClass>
- 45. Why does it need a test phase?
- 46. If test phase not specified when run unit test, integration phase is also run. $ cat test/unit/test/app/sample/FooTests.groovy .... class FooTests { void testSomething() { // do nothing } } $ time grails test-‐app FooTests | Packaging Grails application..... | Tests PASSED -‐ view reports in /xxx/test-‐reports real 0m15.375s user 0m35.119s sys 0m1.858s Oops!
- 47. Integration
- 48. phase
- 49. running!?
- 50. $ time grails test-‐app unit: FooTests | Completed 1 unit test, 0 failed in 546ms | Tests PASSED -‐ view reports in /xxx/test-‐reports real 0m7.303s user 0m18.068s sys 0m0.909s Less
- 51. than
- 52. a
- 53. half
- 54. of
- 55. previous
- 56. sample! Specifying a phase can avoid running an extra phase Without a phase: real 0m15.375s user 0m35.119s sys 0m1.858s
- 57. Required to run test-app • FQCN of target test class • Test phase of target test class
- 58. By the way
- 59. How do you run a test while writing it? • (A) By IDE support (GGTS, IntelliJ, etc.) • (B) By “grails test-app” as standalone • (C) By “grails test-app” via Interactive Mode • (D) Others
- 60. Interactive Mode • Very very useful • Makes command execution faster because the JVM doesn't have to be restarted for each command • $ grails • TAB completion is available • Since Grails 2.0
- 62. But still there is trouble • It needs frequent switching of windows for use while writing code in IDE/editor • To run a test, you must prepare: • FQCN of target test class • Test phase of target test class
- 64. I’m too tired from such Switching, Copy Paste, etc...
- 65. Improx Plugin Provides the way of using interactive mode from other process via TCP. http://grails.org/plugin/improx
- 67. Improx Plugin • You can run a test • Faster than general IDE supports because using interactive mode • Directly from IDE/editor without complicated operations
- 68. Available Editors/IDEs • IntelliJ IDEA • Eclipse/STS/GGTS • Sublime Text 2 • Vim + QuickRun.vim • and more...
- 70. Grails Improx Shell Environment ≪script≫ improxClient TCP/IP ≪script≫ improxSmartInvoker ≪class≫ InteractiveMode ≪class≫ GrailsConsole ≪command≫ improx-start ≪command≫ improx-stop ≪command≫ improx-install-resources other clients ≪class≫ GrailsScriptRunner echo back delegate run script start stop grails command expand client scripts use
- 71. • As same as interactive mode • install-‐plugin • uninstall-‐plugin • Special built-in commands • create-‐app • quit • stop-‐app • exit • !(shell command) • Using threads complexly • run-‐app Unsupported Commands
- 72. How to Install
- 73. BuildConfig.groovy: plugins { //... compile :improx:0.2 //... } Install Plugin
- 74. Install Client Scripts $ grails improx-‐install-‐resources | Improx resources installed successfully: /xxx/improx-‐resources $ chmod +x improx-‐resources/scripts/*.sh $ mv improx-‐resources $HOME/.improx-‐resources If
- 75. once
- 76. you
- 77. store
- 78. them
- 79. to
- 80. global
- 81. path,
- 83. you
- 84. don't
- 85. need
- 86. the
- 87. preparation
- 88. for
- 89. other
- 90. projects.
- 91. How to Use
- 92. Grails Commands • improx-‐start • improx-‐stop • improx-‐install-‐resources
- 93. improx-‐start / improx-‐stop • improx-‐start • Starts an interactive mode proxy server • improx-‐stop • Stops the interactive mode proxy server which is running
- 94. improx-‐install-‐resources • Installs “improx-resources” directory which has some client scripts into your application project improx-‐resources/ ! scripts # improxClient.groovy # improxClient.sh # improxSmartInvoker.groovy ! improxSmartInvoker.sh
- 95. Client Scripts • improxClient.sh • improxClient.groovy • improxSmartInvoker.sh • improxSmartInvoker.groovy
- 96. improxClient.(sh|groovy) Command • Runs a command in interactive mode via TCP port of improx server • Very simple
- 97. $ improxClient.sh help Executing 'help' via interactive mode proxy... ...... Usage (optionals marked with *): grails [environment]* [options]* [target] [arguments]* ...(snipped)... test-‐app tomcat uninstall-‐plugin upgrade war wrapper $
- 98. improxSmartInvoker.(sh|groovy) FILE_PATH • Invokes any *.groovy file in an appropriate way • Very useful for IDE/editor
- 99. How SMART is it?
- 100. grails test-‐app unit: sample.SampleUnitTests $ improxSmartInvoker.sh /path/to/yourApp/ test/unit/sample/SampleUnitTests.groovy Runs the following command on the interactive mode:
- 101. grails test-‐app integration: sample.SampleIntegTests $ improxSmartInvoker.sh /path/to/yourApp/test /integration/sample/SampleIntegTests.groovy Runs the following command on the interactive mode:
- 102. $ improxSmartInvoker.sh /path/to/yourApp/test /functional/sample/SampleFuncTests.groovy grails test-‐app functional: sample.SampleFuncTests Because
- 103. functional
- 104. test
- 105. doesn’t
- 106. work
- 107. well
- 108. on
- 109. interactive
- 110. mode
- 111. at
- 112. present. Runs the following command as new standalone Grails process:
- 113. $ improxSmartInvoker.sh /path/to/scriptDir/myTribialScript.groovy groovy /path/to/scriptDir/myTribialScript.groovy If
- 114. you've
- 115. installed
- 116. GroovyServ,
- 117. the
- 118. groovyclient
- 119. is
- 120. automatically
- 121. used
- 122. instead
- 123. of
- 124. groovy
- 125. command. Runs the following command as a normal Groovy script:
- 126. All you have to do is to set improxSmartInvoker as External Tool of IDE/editor http://kobo.github.io/grails-improx/guide/integrationWithEditors.html
- 127. Simple Protocol
- 128. TCP • Request: COMMAND • Whole response is result of the command • You can also use as client • e.g. telnet, nc
- 129. $ telnet localhost 8096 Trying ::1... Connected to localhost. Escape character is '^]'. help ...... Usage (optionals marked with *): grails [environment]* [options]* [target] [arguments]* ...(snipped)... uninstall-‐plugin upgrade war wrapper $
- 130. HTTP • Request GET /COMMAND HTTP/[0-‐9.]+ • Whole response is result of the command • You can also use as client • e.g. Web browser, wget, curl
- 132. • GroovyServ • http://kobo.github.io/groovyserv/index.html • https://github.com/kobo/groovyserv • Improx - Interactive Mode Proxy • http://kobo.github.io/grails-improx/guide/ • http://grails.org/plugin/improx • https://github.com/kobo/grails-improx • Demo code • https://github.com/nobeans/gr8confeu2013-demo
- 133. Appendix
- 135. Under the hood • Client-Server Communication • Access Control • Propagation of CLASSPATH • Propagation of Environment Variables • Handling System#exit() • Dynamic CWD
- 138. Transfer of Standard In/Out/Err Stream
- 140. $ groovyclient -‐e println ‘Hello, GroovyServ’
- 141. groovyclient ClientConnection OutputStream ≪script≫ println “Hello, GroovyServ!” ≪System.out≫ DynamicDelegatedPrintStream writes data converted based on a protocol into output stream of socket Hello, GroovyServ! if Channel header == ”err” To STDERR if Channel header == ”out” To STDOUT Client console
- 143. $ groovyclient -‐e System.in.eachLine{ println it*5 }
- 144. ClientConnection PipedOutputStream PipedInputStream groovyclient ≪thread≫ StreamRequestHandler ≪script≫ System.in.eachLine { ... } Connected rawData = Socket.inputStream.read() bodyData = convert(rawData) pipedOutputStream.write(bodyData) ≪System.in≫ DynamicDelegatedInputStream
- 145. Protocol over TCP
- 146. InvocationRequest 'Cwd:' cwd LF 'Arg:' arg1 LF 'Arg:' arg2 LF : 'Env:' env1=value1 LF 'Env:' env2=value2 LF : 'Cp:' classpath LF 'AuthToken:' authToken LF LF where: cwd is current working directory. arg1,arg2.. are commandline arguments which must be encoded by Base64. (optional) env1,env2.. are environment variable names which sent to the server. (optional) value1,valeu2.. are environment variable values which sent to the server. (optional) classpath is the value of environment variable CLASSPATH. (optional) authToken is authentication value which certify client is the user who invoked the server. LF is line feed (0x0a, 'n').
- 147. StreamRequest 'Size:' size LF LF body from STDIN where: size is the size of body to send to server. size==-1 means client exited. body from STDIN is byte sequence from standard input.
- 148. InvocationResponse 'Status:' status LF where: status is exit status of invoked groovy script.
- 149. StreamResponse 'Channel:' id LF 'Size:' size LF LF body for STDERR/STDOUT where: id is 'out' or 'err', where 'out' means standard output of the program. 'err' means standard error of the program. size is the size of chunk. body from STDERR/STDOUT is byte sequence from standard output/error.
- 150. Access Control
- 151. Against Evil Request From Evil Client
- 152. Client Address Checking / Authtoken • Request only from loopback address is allowed. • Authtoken is generated and stored into a file when server process starts. • $HOME/.groovy/groovyserver/authtoken-‐PORT • Authtoken always is used for checking that the request is from valid owner user.
- 153. -‐-‐allow-‐from • By default, A request only from loop-back address is available. • You can allow specified addresses. $ groovyserver -‐-‐allow-‐from 192.168.1.1 SECURITY RISK: Be careful when using this option
- 154. -‐-‐authtoken • By default, authtoken is generated automatically. • You can specify any string. SECURITY RISK: Be careful when using this option $ groovyserver -‐-‐authtoken MY_AUTHTOKEN
- 155. At your own risk
- 157. Client and Server are Different Process
- 159. If GroovyServ were too simple, ...
- 161. ≪process≫ groovyserver ≪process≫ groovyclient Please execute this! new foo.Foo().execute() CLASSPATH= /my/classpath/foo.jar
- 162. ≪process≫ groovyserver ≪process≫ groovyclient CLASSPATH= /my/classpath/foo.jar I don’t know its class! Please execute this! new foo.Foo().execute()
- 163. ≪process≫ groovyserver ≪process≫ groovyclient CLASSPATH= /my/classpath/foo.jar I don’t know its class! org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed: script_from_command_line: 1: unable to resolve class foo.Foo @ line 1, column 1. new foo.Foo().execute() ^ 1 error Please execute this! new foo.Foo().execute()
- 164. But GroovyServ is enough smart
- 165. ≪process≫ groovyserver ≪process≫ groovyclient Please execute this! new foo.Foo().execute() CLASSPATH= /my/classpath/foo.jar
- 166. ≪process≫ groovyserver ≪process≫ groovyclient CLASSPATH= /my/classpath/foo.jar Please execute this! new foo.Foo().execute() CLASSPATH= /my/classpath/foo.jar new foo.Foo().execute()
- 167. ≪process≫ groovyserver ≪process≫ groovyclient CLASSPATH= /my/classpath/foo.jar You got it! Please execute this! new foo.Foo().execute() CLASSPATH= /my/classpath/foo.jar
- 168. CLASSPATH which is propagated from client is Volatile
- 170. ≪process≫ groovyserver ≪process≫ groovyclient CLASSPATH= /my/classpath/foo.jar OK Please execute this! new foo.Foo().execute() CLASSPATH= /my/classpath/foo.jar
- 172. ≪process≫ groovyserver ≪process≫ groovyclient CLASSPATH= /my/classpath/bar.jar Please execute this! new bar.Bar().execute() CLASSPATH= /my/classpath/bar.jar OK
- 174. How to specify CLASSPATH from Client
- 175. -classpath / -cp / --classpath $ groovyclient -‐cp /my/classpath/foo.jar -‐e “new foo.Foo().execute()”
- 176. CLASSPATH env $ export CLASSPATH=/my/classpath/foo.jar $ groovyclient -‐e “new foo.Foo().execute()”
- 178. CLASSPATH env $ export CLASSPATH=/my/classpath/foo.jar $ groovyserver
- 179. ≪process≫ groovyserver I feel some power... CLASSPATH= /my/classpath/foo.jar
- 180. ≪process≫ groovyserver ≪process≫ groovyclient CLASSPATH= /my/classpath/foo.jar Please execute this! new foo.Foo().execute() CLASSPATH= /my/classpath/foo.jar I already know foo.jar!
- 182. ≪process≫ groovyserver ≪process≫ groovyclient CLASSPATH= /my/classpath/bar.jar Please execute this! new bar.Bar().execute() CLASSPATH= /my/classpath/bar.jar OK CLASSPATH= /my/classpath/foo.jar
- 186. ≪process≫ groovyserver ≪process≫ groovyclient CLASSPATH= /my/classpath/xxx.jar How to propagate env KEY_A_1=VALUE_A_1 KEY_A_2=VALUE_A_2 KEY_B=VALUE_B KEY_A_1=VALUE_A_1 KEY_B=VALUE_B KEY_A_2=VALUE_A_2
- 187. groovyclient options • -‐Cenv substr • Passes environment variables of which a name includes specified substr • -‐Cenv-‐all • pass all environment variables • -‐Cenv-‐exclude substr • don't pass environment variables of which a name includes specified substr
- 191. Caution: ENV on server isn’t cleared after each invocation
- 196. Server process must not be terminated, even if a script executes System.exit()
- 197. SecurityManager public class NoExitSecurityManager2 extends NoExitSecurityManager { public void checkExit(final int code) { throw new SystemExitException(code, called System.exit( + code + )); } }
- 198. GroovyMain private boolean run() { try { //... processArgs(...) //... } catch (SystemExitException e) throw e; } catch (Throwable e) { //... } } static void processArgs(...) { //... if (!process(cmd, classpath)) { // Disabled because this causes a secondary disaster //System.exit(1); } //... }
- 199. Dynamic CWD
- 201. $ cd /tmp $ groovyserver -r $ cd /home/nobeans $ cat test.txt Can you read me? $ groovyclient -e ‘println(new File(“test.txt”).text)’ Caught: java.io.FileNotFoundException: test.txt (No such file or directory) ...SNIP... ≪process≫ groovyserver CWD=/tmp I wanna use “test.txt” of current directory. Is it “/tmp/test.txt” ? Hmm, such a file not found.
- 202. CWD of client should be set to server for each invocation. But, ...
- 203. In Java, CWD is thoroughly hidden
- 204. Solution • Setting to “user.dir” system property • It affects File#getAbsolutePath() • Changing CWD of native JVM process • Many classes don’t use “user.dir” to complete an absolute path. • But, JNI is troublesome...
- 205. JNA is easy way to use native API UnixLibC.groovy: interface UnixLibC extends com.sun.jna.Library.Library { int chdir(String dir) int setenv(String envVarName, String envVarValue, int overwrite) } pom.xml: dependency groupIdnet.java.dev.jna/groupId artifactIdjna/artifactId version3.2.2/version /dependency Usage: def libc = com.sun.jna.Native.loadLibrary(c, UnixLibC.class) libc.chdir(“/tmp/foo/bar”)
- 206. $ cd /tmp $ groovyserver -r $ cd /home/nobeans $ cat test.txt Can you read me? $ groovyclient -e ‘println(new File(“test.txt”).text)’ Can you read me? ≪process≫ groovyserver CWD=/tmp I wanna use “test.txt” of current directory. ≪process≫ groovyserver CWD=/home/nobeans Is it “/home/nobeans/test.txt” ? OK, I got it. Yes!