Slides for talk at Microsoft Research about Mobl and Spoofax projects.

1. Software Language
Design & Engineering
Eelco Visser
http://eelcovisser.org
Microsoft Research, Redmond
October 20, 2011

2. Software Engineering
Problem
Machine
Domain
bridging the gap between problem domain and solution domain

3. High-Level Languages
Problem
HLL Machine
Domain
"A programming language is low level when its programs
require attention to the irrelevant." Alan J. Perlis (1982)
HLLs reduce gap

4. Domain-Specific Languages
Problem
DSL HLL Machine
Domain
domain-specific languages support more specialization

5. Software Language Design & Engineering
enable software engineers to
effectively design, implement,
and apply domain-specific
software languages

6. Research: Software Language Engineering
automatically derive efficient,
scalable, incremental compiler +
usable IDE from high-level,
declarativelanguage
definition

7. The Spoofax Language Workbench
creating full featured IDEs for domain-specific languages

8. Research: Software Language Design
systematically design domain-specific
software languages
with optimal tradeoff between
expressivity, completeness, portability,
coverage, and maintainability

9. Software Language Design Case Studies
Mobl: client-side stateful web applications
WebDSL: server-side restful web applications

10. Two Talks
Declaratively Programming
the Mobile Web with Mobl
The Spoofax Language Workbench

11. Declaratively Programming
the Mobile Web with Mobl*
Zef Hemel Eelco Visser
http://zef.me http://eelcovisser.org
* To appear: SPLASH/OOPSLA, October 2011, Portland

12. Divergence in Mobile Platforms
Objective-C Java J2ME/C++
HTML/Javascript Java .NET
Native Applications not Portable

13. Convergence in Mobile Platform
Webkit browser Webkit browser
Webkit browser Webkit browser

14. The Universal Userinterface Engine

15. Mobile Web Architecture

16. Rich Applications
Audio
WebDatabases
Full-screen support
Location information (GPS)
Offline support
Canvas
Accelerator support
Multi-touch

17. Native Applications
Address book
Camera
Compass
File IO
Notifications

18. Software Engineering with JavaScript
annotated HTML imperative Javascript
MVC, No Integration, No Abstraction, Accidental Complexity

19. Late Failures in Web Applications
Zef Hemel, Danny M. Groenewegen, Lennart C. L. Kats, Eelco Visser.
Static consistency checking of web applications with WebDSL. Journal of
Symbolic Computation, 46(2):150-182, 2011.

21. typed declarative
http://www.mobl-lang.org
integrated concise

22. Web Application with Touch

23. Portable Applications

24. Mobl Architecture

25. tipcalculator.mobl
application tipcalculator
import mobl::ui::generic
screen root() {
var amount = 20
var percentage = 10
header("Tip calculator")
group {
item { numField(amount, label="amount") }
item { numField(percentage, label="percentage") }
item { "$" label(Math.round(amount * (1 + percentage/100))) }
}
}

26. Model-View Pattern

27. Task Manager

28. Data Model
entity Task {
name : String (searchable)
done : Bool
due : DateTime
category : Category (inverse: tasks)
tags : Collection<Tag> (inverse: tasks)
}
entity Category {
name : String
tasks : Collection<Task> (inverse: category)
}
entity Tag {
name : String
tasks : Collection<Task> (inverse: tags)
}
HTML5 Data Persistence

29. Logic
entity Task {
...
function postpone(days : Num) {
this.due = DateTime.create(
this.due.getFullYear(),
this.due.getMonth(),
this.due.getDate() + days);
}
function import(user : String, pw : String) {
var tasksJSON =
httpRequest("/export?user="+ user + "&pw=" + pw);
foreach(t in tasksJSON) {
add(Task.fromSelectJSON(t));
}
}
}
statically typed: catch errors early

30. Reactive User Interfaces
screen root() {
var phrase = ""
header("Tasks") {
button("Add", onclick={ addTask(); })
}
searchBox(phrase)
group {
list(t in Task.search(phrase) limit 20){
item {
checkBox(t.done, label=t.name)
}
}
}
}

31. Navigation
screen root() { {
addTask()
var phrase = ""
t = Task()
header("Add") {
header("Tasks") {
button("Done", onclick={
button("Add", onclick={ addTask(); })
} add(t);
searchBox(phrase)
screen return;
group {
})
} list(t in Task.search(phrase) limit 20){
textField(t.name)
item {
datePicker(t.due)
checkBox(t.done, label=t.name)
} }
}
}
}

32. Continuations
screen root() {
button("Ask", onclick={
alert("Hello " + prompt("First name") + " "
+ prompt("Last name"));
})
}
screen prompt(question : String) : String {
var answer = ""
header(question) {
button("Done", onclick={
screen return answer;
})
}
textField(answer)
}

33. Reusable Controls
screen root() {
tabSet([("One", tab1),
("Two", tab2)],
defaultTab="One")
}
control tab1() {
header("Tab 1")
label("This is tab 1")
}
control tab2() {
header("Tab 2")
label("This is tab 2")
}
User Interface Idiom: Tab

34. Tab Set: Higher-Order Control
control tabSet(tabs : [(String,Control)], activeTab : String) {
list((tabName, tabControl) in tabs) {
block(onclick={ activeTab = tabName; },
style=activeTab==tabName ?
activeTabButton : inactiveTabButton) {
label(tabName)
}
}
list((tabName, tabControl) in tabs) {
block(activeTab==tabName ? visibleTab : invisibleTab) {
tabControl()
}
}
}
increase coverage: developers can create abstractions

35. User Interface Idiom: Master Detail
screen root() {
header("Tasks")
masterDetail(
Task.all() order by due desc,
taskItem,
taskDetail)
}
control taskItem(t : Task) {
checkBox(t.done, label=t.name)
}
control taskDetail(t : Task) {
textField(t.name)
datePicker(t.due)
}

36. Master Detail: Higher-Order Control
control masterDetail(items : Collection<?>,
masterItem : Control1<?>,
detail : Control1<?>)
{
group {
list(it in items) {
item(onclick={ detailScreen(it,detail); }) {
masterItem(it)
}
}
}
}
screen detailScreen(it : ?, detail : Control1<?>) {
header("Detail") { backButton() }
detail(it)
}

37. Adaptive Layout

38. applications

39. Mobl Applications

40. GR8 Conference Program

41. mPodder

42. Ken
http://itsneh.com/ken/

43. Mobl Summary
❖ Linguistic integration
★ data model, user interface, styling, application logic
❖ Domain abstractions
★ reduce accidental complexity, platform details
❖ User-defined abstractions
★ data binding
★ reactive programming
★ reusable controls
❖ Model-View
★ automate the controller

44. Mobl IDE: Static Cross-Concern Consistency Checking
Constructed on Spoofax Language Workbench

45. The Spoofax Language Workbench*
Rules for Declarative Specification of Languages and IDEs
Lennart Kats Eelco Visser
http://www.lclnet.nl http://eelcovisser.org
* OOPSLA 2010

46. Research: Software Language Engineering
Automatically derive efficient,
scalable, incremental compiler +
usable IDE from high-level,
declarativelanguage
definition

47. Language Definitions for Compilers
Syntax definition
★ concrete syntax
★ abstract syntax
Static semantics
★ error checking
★ name resolution
★ type analysis
Model-to-model transformation
★ express constructs in core language
Code generation
★ translate core language models to implementation

48. Editor Services for Modern IDEs
Syntactic Editor Services
★ syntax checking
★ bracket matching
★ syntax highlighting
★ code folding
★ outline view
Semantic Editor Services
★ error checking
★ reference resolving
★ hover help
★ code completion
★ refactoring

49. Syntax Definition with SDF

50. Declarative Syntax Definition
lexical syntax
[a-zA-Z_$][a-zA-Z0-9_]* -> ID
context-free syntax
STRING -> LimitedSetExp {cons("String")}
NUMBER -> LimitedSetExp {cons("Num")}
LimitedSetExp -> Exp
QId -> LimitedExp {cons("Var")}
"(" Exp ")" -> LimitedExp {cons("Brackets")}
Exp BoolMethodId Exp -> Exp {cons("BinMethodCall"), left}
Exp CompareMethodId Exp -> Exp {cons("BinMethodCall"), left}
Exp TermOperatorMethodId Exp -> Exp {cons("BinMethodCall"), left}
Exp OperatorMethodId Exp -> Exp {cons("BinMethodCall"), left}
character-level grammars: integration of lexical and context-free syntax

51. An Interactive Language Workbench
develop and use language in same environment

52. Debugging Ambiguities

53. Declarative Disambiguation
context-free priorities
Exp "." ID -> LimitedExp
> Exp "." ID "(" {NamedExp ","}* ")" -> LimitedExp
> Exp TermOperatorMethodId Exp -> Exp
> Exp OperatorMethodId Exp -> Exp
> Exp CompareMethodId Exp -> Exp
> Exp BoolMethodId Exp -> Exp
> "!" Exp -> Exp
> Exp "?" Exp ":" Exp -> Exp
> LimitedExp Filter+ -> Exp

54. Parsing after Disambiguation

55. Natural and Flexible Error Recovery*
permissive grammar + backtracking + region discovery [*OOPSLA & SLE 2009]

56. Transformation with Stratego

57. Syntactic Normalization and Desugaring
rules
normalize :
FunctionNoReturnType(manno*, name, farg*, stat*) ->
Function(manno*, name, farg*,
SimpleType(QId("mobl", "void")), stat*)
normalize :
IfNoElse(e, block) -> If(e, block, Block([]))
desugar :
ForInferred(lvalue, e, elem*) -> For(lvalue, t, e, elem*)
where GenericType(_, [t]) := <type-of> e

58. CPS Transform
cps-lift-exprs :
[Return(e)|stats] ->
<concat>[stats2, [Return(e2)|<cps-lift-expressions> stats]]
where not(<is-sync> e)
with {| Exp
: stats2 := <cps-lift-expression> e
; e2 := <Exp>
|}

59. Type Analysis
eq-type-of :
String(_) -> SimpleType(QId("mobl", "String"))
eq-type-of :
FieldAccess(e, x) -> t
where Property(_, _, t, _) := <lookup-property> (<type-of> e, x)

60. Type Checking
constraint-error :
t@SimpleType(_) -> (t, $[Type is not defined: [<pp-mobl-type> t]])
where not(<lookup-type> t)
constraint-error :
t@FieldAccess(e, x) -> (t, $[Property [x] not defined])
where <type-of> e
where not(type-of)
origin tracking

61. Semantic Editor Services
editor-analyze:
(ast, path, project-path) -> (ast2, errors, warnings, notes)
with // ...
editor-complete-proposal :
SimpleType(COMPLETION(_)) -> proposals
where
all-types := <get-all-types>;
proposals := <map(type-name-to-proposal); flatten-list> all-types
editor-hover:
(t@SimpleType(_), position, ast, path, project-path) ->
<get-doc> <lookup-type> t2
where t2 := <lookup-node> (position, ast)
editor-resolve:
(t@SimpleType(qid), position, ast, path, project-path) -> target
where target := <ensure-origin(lookup-type|qid)> t

62. Editor Services Bindings
module MoBL-Builders
imports MoBL-Builders.generated
builders
provider : include/mobl.jar
observer : editor-analyze
on save : generate-artifacts
builder : "Show ATerm" = generate-aterm (openeditor) (realtime) (meta)
builder : "Format code" = format-code (openeditor) (realtime)
builder : "Desugar" = editor-desugar (openeditor) (realtime) (meta)
Connect transformations to IDE

63. Integrated Language Definition Testing*
Includes testing of the editor services [* OOPSLA 2011]

64. Software Language Design & Engineering
Linguistic abstraction:
capture software knowledge in
domain-specific languages http://spoofax.org
separation of concerns + linguistic integration http://mobl-lang.org
cross concern consistency checking http://webdsl.org
early detection of failures http://researchr.org
Language workbench: DSL design and http://eelcovisser.org
implementation with less effort than
traditional language engineering
http://researchr.org/search/publication/mobl+spoofax+webdsl

65. behind the scenes

66. synchronous programming
var results = Task.all().list();
for(var i = 0; i < results.length; i++) {
alert(results[i].name);
}

67. render page
time
query database
and process
results
...

68. render page
time
query database
browser freeze and process
results
...

69. render page
send query
time
...
process query
result
...

70. asynchronous programming
Task.all.list(function(results) {
for(var i = 0; i < results.length; i++) {
alert(results[i].name);
}
});
...

71. Task.all().list(function(results) {
alert("Hello, ");
}); breaks sequential
alert("world!");
execution assumption

72. Task.all().list(function(results) {
// make changes
...
persistence.flush(function() {
alert("Changes saved!");
});
});

73. continuation-passing style
transformation

74. function displayLocationAndReturn() : Coordinates {
var position = mobl::location::getPosition();
log("Lat: " + position.latitude);
log("Long: " + position.longitude);
return position;
}
function displayLocationAndReturn(callback) {
mobl.location.getPosition(function(position) {
console.log("Lat: " + position.latitude);
console.log("Long: " + position.longitude);
callback(position);
});
};

75. function displayLocationAndReturn() : Coordinates {
var position = mobl::location::getPosition();
log("Lat: " + position.latitude);
log("Long: " + position.longitude);
return position;
}
function displayLocationAndReturn(callback) {
mobl.location.getPosition(function(position) {
console.log("Lat: " + position.latitude);
console.log("Long: " + position.longitude);
callback(position);
});
};

76. function displayLocationAndReturn() : Coordinates {
var position = mobl::location::getPosition();
log("Lat: " + position.latitude);
log("Long: " + position.longitude);
return position;
}
function displayLocationAndReturn(callback) {
mobl.location.getPosition(function(position) {
console.log("Lat: " + position.latitude);
console.log("Long: " + position.longitude);
callback(position);
});
};