Ruby AST Tools

Ruby AST Tools
The ParseTree Family

Brian Landau
Wednesday, March 25, 2009

The Ruby AST

Code Code
Style 1 Style 2

AST

Code Code
Style 3 Style 4

The Ruby AST Nodes


The Ruby AST Nodes

def (a,b) def (a,b)

c= a.to_s c= a.to_s

return end return end

def (a,b) def (a,b)

c= a.to_s c= a.to_s

Parsing Ruby in Ruby

ParseTree
ruby_parser


ParseTree


ParseTree

RubyInline based


ParseTree

RubyInline based
Input sources:
Class
Method
Proc
String

ruby_parser


ruby_parser

Uses Racc


ruby_parser

Uses Racc
Retrives line numbers


ruby_parser

Uses Racc
Slower than ParseTree


ruby_parser

Uses Racc
Slower than ParseTree
Sexp output is ParseTree
compatible

Sexp Nodes


Sexp Nodes
class Simple
def say(*args)
puts args * ' '
end
end


Sexp Nodes
class Simple
def say(*args)
puts args * ' '
end
end

s(:class,
:Simple,
nil,
s(:scope,
s(:defn,
:say,
s(:args, :quot;*argsquot;),
s(:scope,
s(:block,
s(:call,
nil,
:puts,
s(:arglist,
s(:call, s(:lvar, :args), :*, s(:arglist, s(:str, quot; quot;))))))))))


Sexp Objects

Inherits from Array
`structure` method
`each_of_type` method
`sexp_type` method
`sexp_body` method


SexpProcessors
Or How I Learned to Love ParseTree


Making a SexpProcessors



Subclass `SexpProcessor`



Call `super` from initialize



Deﬁne `process_xxx` methods



Shift off Sexp nodes



Default process method



Default process method
`rewrite_xxx` methods


Subclass SexpProcessor

class MyProcessor < SexpProcessor

end



Call “super” in “initialize”

def initialize
super
self.warn_on_default = false
self.strict = false
end
end



# ...
Process “call” node
def process_call(exp)
recv = process exp.shift
name = exp.shift
args = process exp.shift

return s()
end
end



# ...

name = exp.shift

return s()
end
Must shift o all Sexp Nodes
end



class MyProcessor SexpProcessor
# ...

name = exp.shift

return s()
end
Must return a Sexp
end



# ...

def default_process(exp)
until exp.size == 0
exp.shift
end
return s()
A default process method
end
end


UniﬁedRuby

pt = ParseTree.new
parse_tree = pt.parse_tree(Simple)
ast = pt.process(parse_tree)

# OR

ast = pt.process(File.read('simple.rb'))


UniﬁedRuby

include UniﬁedRuby

# ...

end


Ruby2Ruby

Ruby2Ruby.translate(Simple, :say)
# = quot;def say(*args)...quot;


ParseTree extensions

require 'parse_tree'
require 'ruby2ruby'
require 'parse_tree_extensions'

bomb = proc do
quot;BOOM!quot;
end

puts bomb.to_ruby # = quot;proc { quot;BOOM!quot; }quot;


ParseTree Family

Flog ambition

Parse
Tree

roodi


ParseTree Family

Code Analytics
Custom DSL


Flog

Takes a set of ﬁles as input
Processes for “bad code”


Flog

Decends from SexpProcessor
Includes UniﬁedRuby


Flog

Assigns weighted scores to:
Node Types
Node Constructions
Method calls


Flog

Reports scores by:
Class
Method


Flog
flog lib
377.6: flog total
11.1: flog/method average

150.2: ClassMethods#acts_as_markup
24.7: HTML#array_to_html
21.5: ClassMethods#get_markdown_class
18.4: main#none
17.9: ActsAsMarkup#none


Flog
ﬂog -g lib
377.6: ﬂog total

171.7: ClassMethods total
21.5: ClassMethods#get_markdown_class

24.7: HTML total
24.7: HTML#array_to_html
...


Flog
flog -d lib
377.6: flog total

50.9: []
42.6: branch
24.0: send
16.5: define_method
11.9: to_s
11.8: assignment
...


self-ﬂagellation

Choose your own weightings


roodi
Ruby Object Oriented Design Inferometer


roodi

Cyclomatic complexity


roodi

Assignment in conditional


roodi

Class/Method line count


roodi

Empty rescue body


roodi

Empty rescue body
No for loops


roodi

Empty rescue body
No for loops
Number of parameters


roodi

sudo gem install roodi
roodi “lib/**/*.rb”


roodi:
How it works


roodi:
How it works

Parse ﬁles


roodi:
How it works

Parse ﬁles
Move node-by-node


roodi:
How it works

Parse ﬁles
Move node-by-node
Evaluate each “Check”


roodi:
How it works

Parse ﬁles
Move node-by-node
Evaluate each “Check”
Do Recursively


Ambition


Ambition

DSL for Querying as Ruby


Ambition

Framework for API Adapters


Ambition

Framework for API Adapters
Generator for new Adapters


Ambition ActiveRecord Adapter

Queries only run when needed
to_s = SQL
to_hash = ﬁnd-compatible hash


Ambition:
How it works

Processors parameters:
Context/Owner
Block


Ambition:
How it works

Context/Owner

User.select { |m| m.name == 'jon' }

Processor Block


Ambition:
How it works

Parse the block
Process nodes recursively
Translate “clauses” to query
string


Ambition:
How it works

Adapters:
Translator for Processors
Make API calls with a Query
Class


The Future:
Ruby 1.9

YARV
A virtual machine based stack architecture
implementation

Ripper
Offers similar functionality as ParseTree and
SexpProcessor


Ripper

Effects YARV behavior
Returns array of arrays literals
Line and column for each token
Ripper::Filter


Ripper:
Lexer
require 'ripper'
require 'pp'

p Ripper.lex(quot;def m(a) nil endquot;)
#= [[[1, 0], :on_kw, quot;defquot;],
# [[1, 3], :on_sp, quot; quot; ],
# [[1, 4], :on_ident, quot;mquot; ],
# [[1, 5], :on_lparen, quot;(quot; ],
# [[1, 6], :on_ident, quot;aquot; ],
# [[1, 7], :on_rparen, quot;)quot; ],
# [[1, 8], :on_sp, quot; quot; ],
# [[1, 9], :on_kw, quot;nilquot;],
# [[1, 12], :on_sp, quot; quot; ],
# [[1, 13], :on_kw, quot;endquot;]]


Ripper:
Sexp Tree

require 'ripper'
require 'pp'

pp Ripper.sexp(quot;def m(a) nil endquot;)
#= [:program,
# [:stmts_add,
# [:stmts_new],
# [:def,
# [:@ident, quot;mquot;, [1, 4]],
# [:paren, [:params, [[:@ident, quot;aquot;, [1, 6]]], nil, nil, nil]],
# [:bodystmt,
# [:stmts_add, [:stmts_new], [:var_ref, [:@kw, quot;nilquot;, [1, 9]]]],
# nil,
# nil,
# nil]]]]


Ripper:
Filter
require 'ripper/ﬁlter'

class CommentStripper Ripper::Filter
def self.strip(src)
new(src).parse
end

def on_default(event, token, data)
print token
end

def on_comment(token, data)
puts
end
end

CommentStripper.strip(ARGF)

QA
Rate at:
http://speakerrate.com/talks/594-ruby-ast-tools

brian.landau@viget.com
http://www.viget.com/extend
http://www.websideattractions.com/

Ruby AST Tools

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a Ruby AST Tools

Semelhante a Ruby AST Tools (20)

Último

Último (20)

Ruby AST Tools

Notas do Editor