The document describes the compilation process of Erlang code to Core Erlang and machine code. It begins with Erlang source code, which is passed through a scanner, parser and various optimizations to produce an intermediate representation in Core Erlang. Core Erlang code is then passed through additional optimizations to produce Erlang Abstract Syntax Machine (ASM) code. The ASM code is finally compiled to machine code. Core Erlang is designed to be a target language for compilers with a simple, functional semantics to facilitate code analysis tools. Joxa code also compiles to Core Erlang through a similar process.

1. JOXA
A Full Featured Lisp on the Erlang VM
Monday, June 3, 13

2. How Compilers Work
-module(test).
-export([hello_world/0]).
hello_world() ->
io:format("Hello World").
Something Like This (That A Human Can Understand):
Monday, June 3, 13

3. How Compilers Work
Becomes Something Like This (That a Machine Can Understand):
00000b0 7453 5472 0000 0000 6d49 5470 0000 2800
00000c0 0000 0300 0000 0300 0000 0400 0000 0100
00000d0 0000 0600 0000 0700 0000 0100 0000 0600
00000e0 0000 0700 0000 0200 7845 5470 0000 2800
00000f0 0000 0300 0000 0500 0000 0100 0000 0600
0000100 0000 0500 0000 0000 0000 0400 0000 0200
0000110 0000 0000 0000 0200 694c 5474 0000 2300
0000120 0000 1700 9c78 6063 6060 6064 e060 ce6f
0000130 e066 48f6 c9cd 57c9 cf08 ca2f 0149 2700
0000140 0526 0026 6f4c 5463 0000 0400 0000 0000
0000150 7441 7274 0000 2800 6c83 0000 0100 0268
0000160 0064 7603 6e73 006c 0000 6e01 0010 b64a
0000170 6c18 9101 034d 1cdb 0854 48dc f414 6a6a
0000180 4943 666e 0000 a600 6c83 0000 0400 0268
0000190 0064 6f07 7470 6f69 736e 006c 0000 6801
00001a0 6402 0600 756f 6474 7269 006b 2f1d 6f68
00001b0 656d 652f 656d 7272 7469 2f74 6f77 6b72
Monday, June 3, 13

4. Compiler Passes
Source
Code
Scanner Parser
IR
Optimizer
Intermediate
Representation
(IR)
Semantic
Analyzer
Machine Code
Generator
Machine
Code
Monday, June 3, 13

5. Erlang Passes
Source
Code
Scanner Parser
Monday, June 3, 13

6. Erlang Passes
-module(test).
-export([hello_world/0]).
hello_world() ->
io:format("Hello World").
Start With This
Monday, June 3, 13

7. Erlang Passes
[{attribute,1,module,test},
{attribute,2,export,[{hello_world,0}]},
{function,2,hello_world,0,
[{clause,2,[],[],
[{call,3,
{remote,3,{atom,3,io},{atom,3,format}},
[{string,3,"Hello World"}]}]}]}]
And End With This:
Monday, June 3, 13

8. Erlang Passes
Core
Erlang
Erlang
ASM
Machine
Code
Monday, June 3, 13

9. Erlang Passes
Core Erlang:
{c_module,[],
{c_literal,[],test},
[{c_var,[],{hello_world,0}},
{c_var,[],{module_info,0}},
{c_var,[],{module_info,1}}],
[],
[{{c_var,[],{hello_world,0}},
{c_fun,[2,{file,[]}],
[],
{c_call,[3,{file,[]}],
{c_literal,[3,{file,[]}],io},
{c_literal,[3,{file,[]}],format},
[{c_literal,[3,{file,[]}],"Hello World"}]}}},
.......
Monday, June 3, 13

10. Erlang Passes
Erlang ASM:
{test,[{hello_world,0},{module_info,0},{module_info,1}],
[],
[{function,hello_world,0,2,
[{label,1},
{line,[{location,[],2}]},
{func_info,{atom,test},{atom,hello_world},0},
{label,2},
{move,{literal,"Hello World"},{x,0}},
{line,[{location,[],3}]},
{call_ext_only,1,{extfunc,io,format,1}}]},
{function,module_info,0,4,
[{label,3},
{line,[]},
{func_info,{atom,test},{atom,module_info},0},
{label,4},
{move,{atom,test},{x,0}},
{line,[]},
{call_ext_only,1,{extfunc,erlang,get_module_info,1}}]},
{function,module_info,1,6,
[{label,5},
{line,[]},
{func_info,{atom,test},{atom,module_info},1},
{label,6},
{move,{x,0},{x,1}},
{move,{atom,test},{x,0}},
{line,[]},
{call_ext_only,2,
{extfunc,erlang,get_module_info,...}}]}],
7}
Monday, June 3, 13

11. Erlang Passes
Erlang ASM:
{test,[{hello_world,0},{module_info,0},{module_info,1}],
[],
[{function,hello_world,0,2,
[{label,1},
{line,[{location,[],2}]},
{func_info,{atom,test},{atom,hello_world},0},
{label,2},
{move,{literal,"Hello World"},{x,0}},
{line,[{location,[],3}]},
{call_ext_only,1,{extfunc,io,format,1}}]},
{function,module_info,0,4,
[{label,3},
{line,[]},
{func_info,{atom,test},{atom,module_info},0},
{label,4},
{move,{atom,test},{x,0}},
{line,[]},
{call_ext_only,1,{extfunc,erlang,get_module_info,1}}]},
{function,module_info,1,6,
[{label,5},
{line,[]},
{func_info,{atom,test},{atom,module_info},1},
{label,6},
{move,{x,0},{x,1}},
{move,{atom,test},{x,0}},
{line,[]},
{call_ext_only,2,
{extfunc,erlang,get_module_info,...}}]}],
7}
Monday, June 3, 13

12. Erlang Passes
Machine Code
00000b0 7453 5472 0000 0000 6d49 5470 0000 2800
00000c0 0000 0300 0000 0300 0000 0400 0000 0100
00000d0 0000 0600 0000 0700 0000 0100 0000 0600
00000e0 0000 0700 0000 0200 7845 5470 0000 2800
00000f0 0000 0300 0000 0500 0000 0100 0000 0600
0000100 0000 0500 0000 0000 0000 0400 0000 0200
0000110 0000 0000 0000 0200 694c 5474 0000 2300
0000120 0000 1700 9c78 6063 6060 6064 e060 ce6f
0000130 e066 48f6 c9cd 57c9 cf08 ca2f 0149 2700
0000140 0526 0026 6f4c 5463 0000 0400 0000 0000
0000150 7441 7274 0000 2800 6c83 0000 0100 0268
0000160 0064 7603 6e73 006c 0000 6e01 0010 b64a
0000170 6c18 9101 034d 1cdb 0854 48dc f414 6a6a
0000180 4943 666e 0000 a600 6c83 0000 0400 0268
0000190 0064 6f07 7470 6f69 736e 006c 0000 6801
00001a0 6402 0600 756f 6474 7269 006b 2f1d 6f68
00001b0 656d 652f 656d 7272 7469 2f74 6f77 6b72
Monday, June 3, 13

13. Core Erlang
Designed as a ‘target language’ for compilers
Strict, higher-order functional language with clear and simple
semantics
Should be as regular as possible to facilitate development of code
walking tools
Translation from Erlang to Core Erlang should be as
straightforward
Should have a human readable textual representation
Monday, June 3, 13

14. Core - The Language Designer
Erlang compiles to core erlang!
Almost all the optimizations and error checking are in the core
Erlang passes!
Core Erlang as a nice ast building module in cerl.erl
Its easy to target!
Monday, June 3, 13

15. Erlang & Core Erlang
-module(test).
-export([hello_world/0]).
hello_world() ->
io:format("Hello World").
module 'test' ['hello_world'/0]
attributes []
'hello_world'/0 =
fun () ->
call 'io':'format'
(“Hello World”)
Erlang: Core Erlang:
Monday, June 3, 13

16. Erlang & Core Erlang AST
Erlang: Core Erlang:
[{attribute,1,module,test},
{attribute,2,export,[{hello_world,0}]},
{function,2,hello_world,0,
[{clause,2,[],[],
[{call,3,
{remote,3,{atom,3,io},{atom,3,format}},
[{string,3,"Hello World"}]}]}]}]
{c_module,[],
{c_literal,[],test},
[{c_var,[],{hello_world,0}}],
[],
[{{c_var,[],{hello_world,0}},
{c_fun,[2,{file,[]}],
[],
{c_call,[3,{file,[]}],
{c_literal,[3,{file,[]}],io},
{c_literal,[3,{file,[]}],format},
[{c_literal,[3,{file,[]}],
"Hello World"}]}}},
Monday, June 3, 13

17. Joxa & Core Erlang
(ns test
(require io))
(defn+ hello-world ()
(io/format “Hello World”))
module 'test' ['hello-world'/0]
attributes []
'hello-world'/0 =
fun () ->
call 'io':'format'
(“Hello World”)
Joxa: Core Erlang:
Monday, June 3, 13

18. Joxa & Core Erlang AST
Joxa: Core Erlang:
[ns, test,
[require, io]]
[{fun, ‘hello-world’, 0},
[]
[{fun, io, format}
[[H e l l o W o r
l d]]]
{c_module,[],
{c_literal,[],test},
[{c_var,[],{hello_world,0}}],
[],
[{{c_var,[],{hello_world,0}},
{c_fun,[2,{file,[]}],
[],
{c_call,[3,{file,[]}],
{c_literal,[3,{file,[]}],io},
{c_literal,[3,{file,[]}],format},
[{c_literal,[3,{file,[]}],
"Hello World"}]}}},
Monday, June 3, 13

19. Joxa Passes
Source
Code
PEG Based
Scanner
Joxa AST
Core
Erlang
Erlang
ASM
Machine
Code
Monday, June 3, 13

20. What is Joxa
Designed as a Simple, Functional, Lisp (though not as a
Common Lisp)
Designed to be a general purpose, usable language in its own
right
Also designed to be a platform for DSLs
Monday, June 3, 13

21. What is Joxa’s Status
Still many features we want to implement
Better Joxa Layer Warnings
More metadata (apropo, live arguments)
Support for Slime + Swank
Production Quality
Monday, June 3, 13

22. A Quick Introduction
(ns sieve-of-eratosthenes
(require (lists :joxify)
(io :joxify))
(use (joxa-core :as core :only (!=/2))
(erlang :only (rem/2 +/2))))
(defn sieve (v primes)
(case v
([] primes)
((h . t)
(sieve (lists/filter (fn (x)
(!= (rem x h) 0)) t)
(+ primes 1)))))
(defn+ sieve (v)
(sieve (lists/seq 2 v) 1))
Monday, June 3, 13

23. A Quick Intro - Macros
(defmacro+ and (&rest args)
(case args
([x y]
`(erlang/and ~x ~y))
((a . b)
`(erlang/and ~a (joxa-core/and ~@b)))
(arg
arg)))
(defmacro+ or (&rest args)
(case args
([x y]
`(erlang/or ~x ~y))
((a . b)
`(erlang/or ~a (joxa-core/and ~@b)))
(arg
arg)))
Monday, June 3, 13

24. ;; Apply the body to every item in the list. The 'value' part of an
;; expression may be a pattern. If the pattern does not match then
;; the non-matching element is skipped
;; Without Guard
;; (dolist (x (lists/seq 0 1)) (+ x 1))
;; With Guard
;; (dolist (x (when (> x 1)) (lists/seq 0 10)) (+ x 1))
;;
(defmacro+ dolist (binding &rest body)
(case binding
([val items]
(let* (arg (joxa-core/gensym "doseq"))
`(lists/foreach
(fn (~arg)
(case ~arg
~@(if (needs-catchall? val)
[`(~val
~@body) `(_ :ok)]
[`(~val
~@body)]))) ~items)))
([val guard items]
(let* (arg (joxa-core/gensym "dolist"))
`(lists/foreach
(fn (~arg)
(case ~arg
(~val ~guard
~@body)
(_ :ok))) ~items)))
(invalid
(erlang/error {:invalid-binding invalid }))))
Monday, June 3, 13

25. ;; Apply the body to every item in the list. The 'value' part of an
;; expression may be a pattern. If the pattern does not match then
;; the non-matching element is skipped, The result of each body is
;; cated to the list and returned as a new list
;; Without Guard
;; (joxa-lists/map (x (lists/seq 0 1)) (+ x 1))
;; With Guard
;; (joxa-lists/map (x (> x 1) (lists/seq 0 10)) (+ x 1))
(defmacro+ map (binding &rest body)
(let* (catcher (joxa-core/gensym "map")
acc-sym (joxa-core/gensym "map")
arg (joxa-core/gensym "map")
body-gen <BODY GEN SEE ->
(case (body-gen binding)
({fun-body items}
`(lists/reverse
(lists/foldl
(fn (~arg ~acc-sym)
~fun-body) [] ~items))))))
BODY GEN:
(fn (binding1)
(case binding1
([val items]
{`(case ~arg
~@(if (needs-catchall? val)
[`(~val
((do ~@body) . ~acc-sym))
`(_ ~acc-sym)]
[`(~val
((do ~@body) . ~acc-sym))])) items})
([val guard items]
{`(case ~arg
~@(if (needs-catchall? val)
[`(~val
(case ~guard
(:true ((do ~@body) . ~acc-sym))
(:false ~acc-sym))
((do ~@body) . ~acc-sym))
`(_ ~acc-sym)]
[`(~val
(case ~guard
(:true ((do ~@body) . ~acc-sym))
(:false ~acc-sym)))])) items})
(invalid
(erlang/error {:invalid-binding invalid })))))
Monday, June 3, 13

26. Records
Provide a unit of abstraction around Erlang’s record syntax
Getters, Setters, Pattern Decomposition Generated
Much, Much prettier then Erlang Records
Monday, June 3, 13

27. More Information
http://joxa.org/
http://docs.joxa.org
https://groups.google.com/group/erlware-questions
irc://irc.freenode.net/joxa (#joxa on freenode)
Monday, June 3, 13