Clojure talk at Münster JUG

Clojure
A small introduction

Alex Ott

http://alexott.net

M¨nster JUG, July 2010
u

Alex Ott (alexott@gmail .com) Clojure M¨nster JUG, July 2010
u 1 / 40

About me...

Professional software developer & architect
Working with diﬀerent programming languages – C/C++, Lisps,
Haskell, Erlang, ...
About 20 years of Lisp experience, including commercial projects
Development on Unix/Linux platforms
Author of articles on programming (including Clojure), Emacs, etc.
Participate in many open source projects, including Incanter, labrepl,
etc.
Maintainer of “Planet Clojure”

u 2 / 40

Outline

1 What is Clojure?

2 Language basics

3 Interoperability with Java

4 Concurrent programming

5 Clojure in real life

6 Sources of information

7 Examples

u 3 / 40

What is Clojure?
Lisp-like language, created by Rich Hickey and
announced in 2007th
Designed to work on existing platforms
Functional programming language, with
immutable data
Special features for concurrent programming
Open sourced with liberal license
Already used in many projects, including
commercial

u 4 / 40

Why new language?
Lisp, but without compatibility with previous
versions
Immutable data and more support for pure
functional programming comparing with other Lisps
Support for concurrent programing inside language
Better integration with target platforms:
JVM & .Net

u 5 / 40

Main features

Dynamically typed language
Very simple syntax, like in other Lisps (code as data)
Support for interactive development
Designed in terms of abstractions
Metaprogramming
Multimethods and protocols (in version 1.2)
Compiled into byte code of target platforms
Access to big number of available libraries

u 6 / 40

Diﬀerences from other Lisps

Changes in syntax, less parentheses
Changes in naming
More ﬁrst-class data structures – maps, sets, vectors
Immutable, by default, data
Ability to link meta-information with variables and functions
“Lazy” collections
There is no reader macros
Case-sensitive names
Lack of tail call optimization (JVM’s limitation) – explicit loops
loop/recur
Exceptions instead of signals and restarts

u 7 / 40

Base data types

Integer numbers of any size – 14235344564564564564
Rational numbers – 26/7
Real numbers – 1.2345 and BigDecimals – 1.2345M
Strings (String class from Java) – "hello world"
Characters (Character from Java) – a, b, . . .
Regular expressions – #"[abc]*"
Boolean – true & false
nil – the same as null in Java, not an empty list as in Lisp
Symbols – test, var1, . . .
Keywords – :test, :hello, . . .

u 8 / 40

Data structures
Separate syntax for diﬀerent collections:
Lists – (1 2 3 "abc")
Vectors – [1 2 3 "abc"]
Maps – {:key1 1234 :key2 "value"}
Sets – #{:val1 "text" 1 2 10}
Sequence – abstraction to work with all collections (including classes
from Java/.Net)
Common set of functions to work with sequences
Lazy operations on sequences
Vectors, maps, and sets are functions – to simplify access to data in
them
Persistent collections
Transient collections – performance optimization
defstruct – optimization of map to work with complex data
structures
u 9 / 40

Syntax

Very simple, homoiconic syntax – program is usual data structure
Special procedure (reader) process text and produce data structures
All objects represent themselves except symbols and lists
Symbols link value with “variable”
Lists represent forms, that could be:
Special form – def, let, if, loop, . . .
Macros
Function call or expression, that could be used as function
(maps, keywords, returned functional values, . . . )
To get list as-is you need to “quote” it (with quote or ’)

u 10 / 40

Code example

( defn f i b o
( [ ] ( concat [1 1] ( f i b o 1 1)))
([ a b]
( l e t [ n (+ a b ) ]
( l a z y −s e q ( c o n s n ( f i b o b n ) ) ) ) ) )
( t a k e 100000000 ( f i b o ) )

( defn vrange2 [ n ]
( loop [ i 0
v ( transient [])]
( i f (< i n )
( recur ( inc i ) ( conj ! v i ))
( persistent ! v ))))

u 11 / 40

Life cycle of Clojure program
Code is read from ﬁle, or REPL, or from other code
Reader transforms program into data structures
Macros are expanded
Resulting code is evaluated/compiled
Produced bytecode is executed by JVM

File
Evaluator/
bytecode JVM
Code Data Compiler
structures

Reader Data Data
structures structures

Code Execution
Macroexpand

Developer
(REPL)

u 12 / 40

Functions

Functions are first-class objects
Function can have variable number of arguments
Function’s definition: defn – top-level functions, fn – anonymous
functions
Simplified syntax for anonymous functions – #(code). For example:
(map #(.toUpperCase %) ["test" "hello"])
(map #(vector %1 %2) [1 2 3] [4 5 6])
You can specify tests & pre/post-conditions. For example,
( d e f n c o n s t r a i n e d −s q r [ x ]
{ : p r e [ ( pos ? x ) ] : p o s t [( > % 1 6 ) , (< % 2 2 5 ) ] }
(∗ x x ) )

Each function is compiled into separate Java class
Each function implements Runnable interface

u 13 / 40

Metaprogramming & macros

Macros receives source code and returns new code, that will compiled
Big part of language is written using macros
Variable number of arguments (same as functions)
Code could be generated with list functions, but it’s much handy to
use quasi-quote – ‘ and substitution operators – ~ and ~@
The # suﬃx in names is used to generate unique names
macroexpand-1 & macroexpand are used to debug macros

u 14 / 40

Macros examples (1)

The standard macros when:
( d e f m a c r o when
[ t e s t & body ]
( l i s t ’ i f t e s t ( c o n s ’ do body ) ) )
when used as:
( when ( pos ? a )
( p r i n t l n " p o s i t i v e ") (/ b a ) )
will expanded into:
( i f ( pos ? a )
( do
( p r i n t l n " p o s i t i v e ")
(/ b a ) ) )

u 15 / 40

Macros examples (2)

The standard macros and
( d e f m a c r o and
( [ ] true )
([ x] x)
( [ x & next ]
‘ ( l e t [ and# ~x ]
( i f and# ( and ~@next ) and # ) ) ) )
will expanded into diﬀerent code, depending on number of arguments:
user> ( macroexpand ’ ( and ) ) == t r u e
>
user> ( macroexpand ’ ( and (= 1 2 ) ) ) == (= 1 2 )
>
user> ( macroexpand ’ ( and (= 1 2 ) (= 3 3 ) ) ) ==>
( let∗ [ and__4457__auto__ (= 1 2)]
( if and__4457__auto__
( c l o j u r e . c o r e / and (= 3 3))
and__4457__auto__ ) )
u 16 / 40

Polymorphism & multimethods

Extensibility
Multimethods aren’t bound exclusively to types/classes
Dispatching is performed using results of user-specified dispatch
function
Dispatching on several parameters
Differs from CLOS – absence of :before, :after, . . .
defmulti – is analog of defgeneric in Common Lisp
Defined as ( defmulti func−name dispatch−fn) + set of
implementations (via defmethod)
You can also define hierarchical relationships with derive, and use
them in dispatch (via isa?)

u 17 / 40

Multimethods examples (1)

Simple dispatching using data type/class:
−e
( d e f m u l t i m xample c l a s s )
−e
( d e f m e t h od m x a m p l e S t r i n g [ t h i s ]
( p r i n t l n " This i s s t r i n g ’" t h i s " ’"))
−e
( d e f m e t h od m x a m p l e j a v a . u t i l . C o l l e c t i o n [ t h i s ]
( p r i n t " This i s c o l l e c t i o n ! " ) )
and we’ll get:
−e
(m x a m p l e " H e l l o " ) == " T h i s i s s t r i n g ’ H e l l o ’ "
>
−e
(m x a m p l e [ 1 2 3 ] ) == " T h i s i s c o l l e c t i o n ! "
>
−e
(m x a m p l e ’ ( 1 2 3 ) ) == " T h i s i s c o l l e c t i o n ! "
>

u 18 / 40

Multimethods examples (2)

( defmulti encounter
( fn [ x y ] [ ( : Species x ) ( : Species y ) ] ) )
( d e f m e t h od e n c o u n t e r [ : Bunny : L i o n ] [ b l ] : run−away )
( d e f m e t h od e n c o u n t e r [ : L i o n : Bunny ] [ l b ] : e a t )
( d e f m e t h od e n c o u n t e r [ : L i o n : L i o n ] [ l 1 l 2 ] : f i g h t )
( d e f m e t h od e n c o u n t e r [ : Bunny : Bunny ] [ b1 b2 ] : mate )
( d e f b1 { : S p e c i e s : Bunny } )
( d e f b2 { : S p e c i e s : Bunny } )
( def l 1 { : S p e c i e s : Lion })
( def l 2 { : S p e c i e s : Lion })

( encounter b1 b2 ) ==> : mate
( encounter b1 l1 ) ==> : run−away
( encounter l1 b1 ) ==> : eat
( encounter l1 l2 ) ==> : fight

u 19 / 40

Protocols & Datatypes

Introduced in version 1.2 (will released shortly)
Much faster than multimethods
Allows to write “Clojure in Clojure”
Dispatching is done only on data types
Similar to type classes in Haskell
Java interface is created for each protocol
defrecord & deftype deﬁne new data types
extend-protocol & extend-type bind protocol(s) with data types
(not only with deﬁned in Clojure!)
reify is used to implement protocols and interfaces for “once-used
types”

u 20 / 40

Protocol’s examples

( d e f p r o t o c o l H e l l o " Test of p r o t o c o l "
( h e l l o [ t h i s ] " h e l l o function "))

( d e f r e c o r d B [ name ] H e l l o
( h e l l o [ t h i s ] ( s t r " H e l l o " ( : name t h i s ) " ! " ) ) )
( h e l l o (B . " U s e r " ) ) == " H e l l o U s e r ! "
>

( e x t e n d −p r o t o c o l H e l l o S t r i n g
( hello [ this ] ( str " Hello " this "!")))
( h e l l o " w o r l d " ) == " H e l l o w o r l d ! "
>

( e x t e n d −p r o t o c o l H e l l o j a v a . l a n g . O b j e c t
( h e l l o [ t h i s ] ( s t r " H e l l o ’" t h i s
" ’ ! (" ( type t h i s ) " ) " ) ) )
( h e l l o 1 ) == " H e l l o ’ 1 ’ ! ( c l a s s j a v a . l a n g . I n t e g e r ) "
>

u 21 / 40

Miscellaneous
Metadata – #^{} in 1.0 & 1.1, or simply ^{} in 1.2
Optional type speciﬁcation (type hints) – #^Type or ^Type
You can specify tests directly in function’s declaration
Scope management
Access and change of metadata from code
Don’t change value equality
Namespaces:
are ﬁrst-class objects
are used to organize code into libraries
Data destructuring (function arguments, etc.)
( l e t [ [ a b & c : as e ] [1 2 3 4 ] ] [ a b c e ] )
== [ 1 2 ( 3 4 ) [ 1 2 3 4 ] ]
>
( l e t [ { : k e y s [ a b c ] : a s m : o r {b 3}} { : a 5 : c 6 } ]
[ a b c m] ) == [ 5 3 6 { : a 5 : c 6 } ]
>

u 22 / 40

Interoperability with Java
Two-way interoperability with target platform:
Creation of instances – new or Class.
Call of Java methods – ., .., doto
Class & interface generation – gen-class, gen-interface, or proxy
(anonymous class)
You can execute Clojure code in Java programs
Separate functions to work with Java arrays:
make-array – array creation
aget/aset – access to array’s elements
amap/areduce – iteration over array’s elements
memfn allows to use Java methods as arguments of map, filter, etc.
The set! special form to set values inside class
Generation and catch of exception – throw & catch

u 23 / 40

Interoperability examples
Instance creation:
( new j a v a . u t i l . Date ) <== ( j a v a . u t i l . Date . )
>
Call of Java methods and access to data:
( . s u b s t r i n g " H e l l o World " 0 5 ) == " H e l l o "
>
( . " H e l l o World " s u b s t r i n g 0 5 ) == " H e l l o "
>
Math/ PI == 3 . 1 4 1 5 9 2 6 5 3 5 8 9 7 9 3
>
( I n t e g e r / p a r s e I n t " 4 2 " ) == 42
>
.. is used to chained calls:
( . . System g e t P r o p e r t i e s ( g e t " o s . name " ) ) == "Mac
>
System . g e t P r o p e r t i e s ( ) . g e t ( " o s . name " )
doto – call of several methods for one object:
( d o t o ( j a v a . u t i l . HashMap . )
( . p u t " a " 1 ) ( . p u t "b" 2 ) )

u 24 / 40

Concurrent programming
Special language features, that provides data mutation:
Refs – synchronous, coordinated change
Agents – asynchronous, not coordinated
Atoms – synchronous, not coordinated
Vars – local for current thread
@ (deref) is used to get access to data
Parallel code execution
future
pmap, pvalues & pcalls
Native threads
Synchronization with promise

u 25 / 40

State and Identity

Clojure separates concepts of state and identity
State (value) isn’t changed!
More information at http://clojure.org/state

Code Code Code
(past) (present) (future)

Identity
(ref/agent/atom
name)

State S1 State S2 State S3
[1 2 3 4] [1 2 3 4 5 6 7] [?????]

u 26 / 40

Refs & STM

Coordinated change of data from several threads
Software Transaction Memory provides atomicity, consistency, and
isolation
Changes could be made only inside transaction
You can add function for data validation
Ability to add watcher function

( def counters ( r e f {}))
( d e f n get −c o u n t e r [ k e y ]
( @counters key 0))
( d e f n i n c r e m e n t −c o u n t e r [ k e y ]
( dosync
( a l t e r c o u n t e r s a s s o c key
( i n c ( @counters key 0 ) ) ) ) )

u 27 / 40

Agents

Asynchronous change of data – “ﬁre and forget”
Thread pools for data update functions: send – for “fast” functions,
and send-off – for “heavyweight” functions (separate thread pool)
send & send-off get function that will applied to agent’s current
state
Validators and watchers
You can wait until ﬁnish of all updates

( def acounters ( agent {}))
( d e f n i n c r e m e n t −a c o u n t e r [ k e y ]
( send a c o u n t e r s a s s o c key
( i n c ( @counters key 0 ) ) ) )
( d e f n get −a c o u n t e r [ k e y ]
( @acounters key 0))

u 28 / 40

Vars & Atoms
Atoms
Synchronous data change without coordination
Data are changed by applying function to current state of atom
Function could be called several times
Function shouldn’t have side eﬀects!
Vars
Provide data change only in current thread
binding links new values with symbols
Change aﬀects all functions, called from current thread
Be careful with lazy sequences!

( def ∗ var ∗ 5)
( defn foo [ ] ∗ var ∗)
( f o o ) == 5 >
( b i n d i n g [ ∗ v a r ∗ 1 0 ] ( f o o ) ) == 10
>

u 29 / 40

Parallel code execution
future
executes given code in separate thread
@ is used to get results of code execution
@ blocks current thread, if results aren’t available yet
results of code execution are cached
promise
is used for synchronization between parts of programs
deliver sets value of promise
@ reads value, or blocks execution if value isn’t set yet
pmap, pvalues & pcalls are used for “heavyweight” operations, that
could be performed in parallel

u 30 / 40

Clojure in real life
Existing infrastructure:
IDEs
Build tools
Debugging and related tools
Libraries
Code repositories
Clojure/core – commercial support & consulting
Clojure is used in commercial projects since 2008
Sources of information

u 31 / 40

Infrastructure: IDEs and build tools
Supported by most of popular IDEs:
Emacs + SLIME (most popular now)
VimClojure
NetBeans
Eclipse
IntelliJ IDEA
Build tools:
Clojure support in Maven, Ant and Gradle
Leiningen – written in Clojure, extensible and very simple

u 32 / 40

Infrastructure: libraries & repositories

Simple access to big number of existing Java libraries
Clojure-speciﬁc libraries:
Clojure-Contrib – “semi-standard” libraries
Compojure, Ring, Scriptjure – Web development
ClojureQL – databases
Incanter – R-like environment and libraries
other – see at http://clojars.org
Repositories:
build.clojure.org
clojars.org

u 33 / 40

Commercial projects

FlightCaster – machine learning, etc.
Relevance, Inc. – consulting
Runa – marketing services
Sonian Networks – archiving solutions
BackType – social media analytics
DRW Trading Group – trading and ﬁnance
DocuHarvest project by Snowtide Informatics Systems, Inc.
ThorTech Solutions – scalable, mission-critical solutions
TheDeadline project by freiheit.com (http://www.freiheit.com/)
and many more. . .

u 34 / 40

Sources of information (1)
Main sites:
Site of the language – http://clojure.org
Planet Clojure – http://planet.clojure.in
the #clojure IRC channel at freenode.net
The labrepl project (http://github.com/relevance/labrepl) –
learning environment
Try-Clojure (http://www.try-clojure.org/) – you can execute
Clojure code via Web-browser
http://clojuredocs.org/ – documentation and examples
German resources:
Book in German will released in 2010 fall
(http://www.clojure-buch.de/)
clojure-de mailing list
(http://groups.google.com/group/clojure-de)
videos about Clojure in German (http://www.rheinjug.de/videos/
gse.lectures.app/Talk.html#Clojure)

u 35 / 40

Sources of information (2)
Books:
Programming Clojure – 2009th, Clojure v. 1.0
Practical Clojure. The Deﬁnitive Guide – 2010th, Clojure, v. 1.2
The Joy of Clojure (beta)
Clojure in Action (beta)
Clojure Programming on Wikibooks
Clojure Notes on RubyLearning
Video-lectures & screencasts
Clojure user groups around the world
Clojure study courses (on-site in USA & Europe, or online)

u 36 / 40

Examples. Hadoop

Hadoop word count example is only lines of code (using
clojure-hadoop), instead dozens for Java...
All low-level details are handled by macros
( n s c l o j u r e −hadoop . e x a m p l e s . w o r d c o u n t 3
( : import ( java . u t i l StringTokenizer )) )

( d e f n my−map [ k e y v a l u e ]
( map ( f n [ t o k e n ] [ t o k e n 1 ] )
( e n u m e r a t i o n −s e q ( S t r i n g T o k e n i z e r . v a l u e ) ) ) )

( d e f n my−r e d u c e [ k e y v a l u e s −f n ]
[ [ k e y ( r e d u c e + ( v a l u e s −f n ) ) ] ] )

u 37 / 40

Examples. Cascalog

Domain speciﬁc language to query data in Hadoop with joins,
aggregates, custom operations, subqueries, etc.
Interactive work with data, using Clojure
Available from http://github.com/nathanmarz/cascalog
Example: query for persons younger than 30:
(?<− ( s t d o u t ) [ ? p e r s o n ? age ] ( age ? p e r s o n ? age )
(< ? age 3 0 ) )

u 38 / 40

Examples. Concurrency

Ants:
320 lines of code, with commentaries and documentation
Massive parallel execution without explicit threading
Many independent objects
Graphical interface
Cluster analysis with K-means algorithm:
Each cluster represented by agent
Automatically uses all available cores
http://www.informatik.uni-ulm.de/ni/staff/HKestler/
Reisensburg2009/talks/kraus.pdf

u 39 / 40

Thank you

Questions

u 40 / 40

Clojure talk at Münster JUG

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (18)

Semelhante a Clojure talk at Münster JUG

Semelhante a Clojure talk at Münster JUG (20)

Último

Último (20)

Clojure talk at Münster JUG