"Code Generation in Perl" presented to Silicon Valley Perl in Santa Clara CA by Ian Kluft on September 7, 2017. The presentation surveys the landscape of code generation tools and techniques including preprocessers, compliers and model languages, as well as CPAN modules to help with those tasks. Code generation has always been useful for automation of tedious tasks. There is also some debunking of myths from science fiction about what is currently possible with code generation. No matter how much processing is involved, at some level someone still has to write the programs.

1. Code Generation in Perl
Presentation by Ian Kluft
for Silicon Valley Perl
September 7, 2017
Santa Clara, California

2. What is Code Generation?
●
Programs that write programs
●
There are many ways to approach this
●
Advantage: lets the coder work at a higher level
●
Disadvantage: someone still has to write program

3. The simplest example
●
A program hard-coded to print another program
●
Even the simple example has non-trivial quoting
#!/usr/bin/perl
use strict;
use warnings;
print qq(#!/usr/bin/perln);
print qq(use strict;n);
print qq(print "hello worldn";n);

4. Other approaches
●
Templates – fill in the blanks
●
Translators – parse, analyze and re-generate
●
Compilers – parse high level, generate lower level
●
SQL schemas & queries – generate SQL code
●
XML parsing – generate programs from XML data
●
UML tools – draw diagrams, generate code

5. CPAN Modules
which generate code
●
At a low level it’s widespread
– Many modules & scripts generate code for eval()
●
Perl build tools
●
SQL generation for queries and schemas
●
Perl application frameworks
– Easier to get started with tools that generate examples
●
Compilers

6. CPAN Modules
Perl build tools
●
CPAN Modules which generate code
– Dist::Zilla
– ExtUtils::MakeMaker
– Module::Build
●
These generate build scripts from module
configuration data

7. CPAN Modules
SQL code generation
●
DBIx::Class – object relational mapper
●
Fey – SQL schema and query generator
●
…CPAN is huge - the list goes on and on...
●
Many examples – which do you like?

8. CPAN Modules
XML & data code generators
●
Data::Dumper – dumps data as Perl code
●
Faker – extensible fake data generator
●
Not as much code generation from XML on CPAN
– Obvious category for ad hoc tools
– Any XML parser can be used to generate code
●
Writing HTML is not technically writing code
– But it also has similarities

9. CPAN Modules
Compilers
●
B::* - access to Perl compiler
●
Compiler generation tools
– Parse::Yapp/Parse::Eyapp – compiler parser generator
●
Similar to Unix “yacc” but generates Perl code
– Parse::RecDescent – generate recursive descent
compiler
– Parse::Lex – generate lexical analyzer for compiler
●
A lot of CPAN modules claiming to “compile”
actually “translate”

10. How Compilers Work
●
High level view of a compiler
Preprocessor
Scanner/
Tokenizer
Parser Semantics
●
Preprocessor – macros & text manipulation
●
Scanner – generates stream of tokens
– Removes comments
●
Parser – recognizes language structures
●
Semantics – generate code
– Some compilers like Perl make intermediate code for interpretation

11. Using Parser Generator Tools
●
Have you noticed that language compilers do things that
are way above the capabilities of regular expressions?
– RegEx’es are ideal for tokenizers/scanners
– There’s another level of language parsing above that
●
One example is to grab a yacc grammar and translate it
to Parse::Yapp or Parse::Eyapp
– I did that once for a Perl script that parsed /etc/sudoers files for
automated security audits
●
Language definitions in Backus-Naur Form (BNF) can be
converted to language parser defintions

12. UML
Unified Modeling Language
●
XML format representing standard diagrams
– Parts of a program or system design
– Analysis of code can generate UML for documentation
– Active research into ways to generate code from UML
●
Code generation can be done
●
But diagramming is far more tedious than typing code
●
You can experiment with this on your own projects
●
Idea is to use UML as design/documentation & generate code

13. UML – Unified Modeling Language
types of diagrams
Diagram
Class
Diagram
Component
Diagram
Object
Diagram
Activity
Diagram
Use Case
Diagram
Profile
Diagram
Composite
Structure
Diagram
Deployment
Diagram
Package
Diagram
Interaction
Diagram
State
Machine
Diagram
Sequence
Diagram
Communication
Diagram
Interaction
Overview
Diagram
Timing
Diagram Notation: UML
Behaviour
Diagram
Structure
Diagram
UML diagram types – as a class diagram (from Wikipedia)

14. UML – Unified Modeling Language
example class: Pizza
UML code generation makes class
source code starter templates for
●
Pizza
●
CrustEnum
●
ToppingList
●
ToppingEnum
●
SizeEnum
How would you go from here to write the code for the classes in UML?
Good ways to do that are still open to suggestion.

15. UML – Unified Modeling Language
Tools for Code Generation from UML
●
Open source “umbrello”
from KDE generated
previous slide’s graphic
●
Eclipse IDE has UML
generation tools
●
CPAN has many UML
modules, mostly for
generating UML from Perl for
analysis
●
Many other examples...

16. Setting expectations
What a non-engineer
manager wants…
This one doesn’t grasp
the current limitations.
He isn’t alone.
What people want is
often a hint where
research should push
the limits for the future.

17. Setting expectations
”Artificial Intelligence”
●
Artificial Intelligence has a broken promise built right into
its name
●
It has always been advanced algorithms
– It’s always non-trivial
– Not all uses of the term AI are as sophisticated as claimed
●
Its results usually appear clever
●
It can do lots of things people don’t do well
●
But it still can’t think
●
It isn’t real or artificial intelligence… yet

18. Science Fiction examples
Code generation and AI
●
One early SciFi story: “The Adolescence of P1” by
Thomas J Ryan, 1977
●
Based on modem-connected IBM 360 mainframes
●
Program to break OS “supervisor”
– Needed decision trees, code generator, threat analysis
– When told to shut down it played dead, kept running
– AI evolved organically
– Installed itself like a worm, broke OS like a virus
●
Few SciFi AI’s attempted such depth

19. Code Generation
Setting expectations back to reality
●
SciFi myths need to be debunked
●
Code generation just adds a level of abstraction
●
Someone still has to write the code
●
Short of an actual artificial intelligence, programs won’t
be writing programs any time soon
●
Maybe during 21st Century?
– 20th Century showed how much advancement can happen
– But AI is a fundamentally difficult computing challenge
– Don’t count on it

20. Code Generation
Q & A