JavaScript used to be confined to the browser. But these days, it becoming increasingly popular in server-side applications in the form of NodeJS. NodeJS provides event-driven, non-blocking I/O model that supposedly makes it easy to build scalable network application. In this talk you will learn about the consequences of combining the event-driven programming model with a prototype-based, weakly typed, dynamic language. We will share our perspective as a server-side Java developer who wasn’t entirely happy about JavaScript in the browser, let alone on the server. You will learn how to use NodeJS effectively in modern, polyglot applications.
NodeJS: the good parts? A skeptic’s view (jmaghreb, jmaghreb2013)
1. NodeJS: the good parts?
A skeptic’s view
Chris Richardson
Author of POJOs in Action
Founder of the original CloudFoundry.com
@crichardson chris@chrisrichardson.net http://plainoldobjects.com
@crichardson
2. Presentation goal
How a grumpy, gray-haired
server-side Java developer
discovered an appreciation
for NodeJS and JavaScript
@crichardson
8. Agenda
Overview of NodeJS
JavaScript: Warts and all
The Reactor pattern: an event-driven architecture
NodeJS: There is a module for that
Building a front-end server with NodeJS
@crichardson
14. Dynamic and weakly-typed
Dynamic:
Types are associated with values - not variables
Define new program elements at runtime
Weakly typed:
Leave out arguments to methods
Read non-existent object properties
Add new properties by simply setting them
@crichardson
16. JavaScript is a prototypal
language
Person
__proto__
sayHello
...
Prototype
function
...
inherited
Chris __proto__
name
nickname
“Chris”
“CER”
overrides
object specific
@crichardson
17. Prototypal code
Not defined
here
$ node
> var person = { sayHello: function () { console.log("Hello " + this.name); }};
[Function]
> var chris = Object.create(person, {name: {value: "Chris"}});
undefined
> var sarah = Object.create(person, {name: {value: "Sarah"}});
undefined
> chris.sayHello();
Hello Chris
create using
undefined
properties
> sarah.sayHello();
prototype
Hello Sarah
undefined
> chris.sayHello = function () { console.log("Hello mate: " + this.name); };
[Function]
> chris.sayHello();
Hello mate: Chris
undefined
@crichardson
18. JavaScript is Functional
function makeGenerator(nextFunction) {
Return a function
closure
var value = 0;
return function() {
var current = value;
value = nextFunction(value);
return current;
};
Pass function as an
argument
}
var inc = makeGenerator(function (x) {return x + 1; });
> inc()
0
> inc()
1
@crichardson
19. But JavaScript was created
in a hurry
The ‘Java...’ name creates expectations that it can’t satisfy
Fake classes: Hides prototypes BUT still seems weird
global namespace
scope of vars is confusing
Missing return statement = confusion
‘function’ is really verbose
‘this’ is dynamically scoped
Unexpected implicit conversions: 99 == “99”!
truthy and falsy values
52-bit ints
@crichardson
20. Dynamic + weakly-typed (+ event-driven)
code
+
misspelt property names
lots of time spent in the abyss
Essential: Use IDE integrated
with JSLint/JSHint + tests
21. Prototypal languages have
benefits BUT
Developers really like classes
JavaScript prototypes lack the powerful features from
the Self language
e.g. Multiple (and dynamic) inheritance
http://www.cs.ucsb.edu/~urs/oocsb/self/papers/papers.html
@crichardson
22. Verbose function syntax
> var numbers = [1,2,3,4,5]
> numbers.filter(function (n) { return n % 2 == 0; } ).map(function (n) { return n * n; })
[ 4, 16 ]
>
Versus
Prelude> let numbers = [1,2,3,4,5]
Prelude> map (n -> n * n) (filter (n -> mod n 2 == 0) numbers)
[4,16]
Or
scala> val numbers = 1..5
scala> numbers filter { _ % 2 == 0} map { n => n * n }
Vector(4, 16)
@crichardson
23. Verbose DSLs
describe('SomeEntity', function () {
beforeEach(function () { ... some initialization ... });
Jasmine
it('should do something', function () {
...
expect(someExpression).toBe(someValue);
});
});
Versus
class SomeScalaTest ...{
Scalatest
before { ... some initialization ... }
it should "do something" in {
...
someExpression should be(someValue)
}
@crichardson
24. JavaScript is the language
of the web
“You have to use the programming
language you have, not the one that you
might want”
@crichardson
25. It works but the result is
lost opportunities
and
impeded progress
@crichardson
26. But if you think that this isn’t
a problem then perhaps ....
“Stockholm syndrome ... is a psychological
phenomenon in which hostages ... have
positive feelings toward their captors,
sometimes to the point of defending them...”
http://en.wikipedia.org/wiki/Stockholm_syndrome
@crichardson
27. Martin Fowler once said:
"...I'm one of those who despairs that a
language with such deep flaws plays such an
important role in computation. Still the
consequence of this is that we must take
javascript seriously as a first-class language
and concentrate on how to limit the damage
its flaws cause. ...."
http://martinfowler.com/bliki/gotoAarhus2012.html
@crichardson
28. Use just the good parts
Douglas
Crockford
http://www.crockford.com/
@crichardson
29. Use a language that
compiles to JavaScript
TypeScript
Classes and interfaces (dynamic structural typing)
Typed parameters and fields
Dart
Class-based OO
Optional static typing
Bidirectional binding with DOM elements
Less backwards compatibility with JavaScript
Also has it’s own VM
@crichardson
30. CoffeeScript Hello World
Classes :-)
http = require('http')
Concise
class HttpRequestHandler
constructor: (@message) ->
Bound method
handle: (req, res) =>
res.writeHead(200, {'Content-Type': 'text/plain'})
res.end(@message + 'n')
handler = new HttpRequestHandler "Hi There from CoffeeScript"
server = http.createServer(handler.handle)
server.listen(1338, '127.0.0.1')
console.log('Server running at http://127.0.0.1:1338/')
@crichardson
33. About the Reactor pattern
Defined by Doug Schmidt in 1995
Pattern for writing scalable servers
Alternative to thread-per-connection model
Single threaded event loop dispatches events on
handles (e.g. sockets, file descriptors) to event handlers
@crichardson
35. Benefits
Separation of concerns - event handlers separated
from low-level mechanism
More efficient - no thread context switching
Simplified concurrency - single threaded = no
possibility of concurrent access to shared state
@crichardson
36. Drawbacks
Non-pre-emptive - handlers must not take a long time
Difficult to understand and debug:
Inverted flow of control
Can’t single step through code easily
Limited stack traces
No stack-based context, e.g. thread locals, exception handlers
How to enforce try {} finally {} behavior?
@crichardson
37. NodeJS app = layers of event
handlers
Recurring
events
from
Event
Emitters
Application code
Event
listener
HTTP
Callback
function
DB driver
...
One time
events:
async
operation
completion
Basic networking/file-system/etc.
NodeJS event loop
@crichardson
38. Async code = callback hell
Scenarios:
Sequential: A
B
C
Scatter/Gather: A and B
C
Code quickly becomes very messy
@crichardson
39. Messy callback code
The result of
getProductDetails
getProductDetails = (productId, callback) ->
productId = req.params.productId
result = {productId: productId}
Propagate error
makeCallbackFor = (key) ->
(error, x) ->
if error
callback(error)
else
result[key] = x
if (result.productInfo and result.recommendations and result.reviews)
callback(undefined, result)
Update result
Gather
getProductInfo(productId, makeCallbackFor('productInfo'))
getRecommendations(productId, makeCallbackFor('recommendations'))
getReviews(makeCallbackFor('reviews'))
Scatter
@crichardson
40. Simplifying code with
Promises (a.k.a. Futures)
Functions return a promise - no callback parameter
A promise represents an eventual outcome
Use a library of functions for transforming and
composing promises
Promises/A+ specification - http://promises-aplus.github.io/promises-spec
when.js (part of cujo.js by SpringSource) is a popular
implementation
Crockford’s RQ library is another option
@crichardson
42. Not bad but lacks Scala’s
syntactic sugar
class ProductDetailsService .... {
def getProductDetails(productId: Long) = {
for (((productInfo, recommendations), reviews) <getProductInfo(productId) zip
getProductInfo(productId) zip
getRecommendations(productId) zip
getRecommendations(productId) zip
getReviews(productId)
getReviews(productId))
yield
ProductDetails(productInfo, recommendations, reviews)
}
}
@crichardson
43. Long running computations
Long running computation
blocks event loop for
other requests
Need to run outside of main event loop
Options:
Community: web workers threads
Built-in: NodeJS child processes
@crichardson
44. Using child processes
parent.js
var child = require('child_process').fork('child.js');
function sayHelloToChild() {
child.send({hello: "child"});
}
Create child process
Send message to child
setTimeout(sayHelloToChild, 1000);
child.on('message', function(m) {
console.log('parent received:', m);
});
function kill() {
child.kill();
}
setTimeout(kill, 2000);
child.js
process.on('message', function (m) {
console.log("child received message=", m);
process.send({ihateyou: "you ruined my life"})
});
@crichardson
45. Modern multi-core machines
vs. single-threaded runtime
Many components of many applications
Don’t need the scalability of the Reactor pattern
Request-level thread-based parallelism works fine
There are other concurrency options
Actors, Software transactional memory, ...
Go goroutines, Erlang processes, ...
Imposing a single-threaded complexity tax on the
entire application is questionable
@crichardson
48. Thousands of community
developed modules
https://npmjs.org/
web frameworks, SQL/NoSQL
database drivers, messaging, utilities...
@crichardson
49. What’s a module?
foo.js
One or more JavaScript files
exports.sayHello = function () {
console.log(“Hello”);
}
Optional native code:
Compiled during installation
JavaScript != systems programming language
Package.json - metadata including dependencies
@crichardson
51. Easy to use
Core module OR
Path to file OR
module in node_modules
Module’s exports
var http = require(“http”)
var server = http.createServer...
@crichardson
53. There is a module for that...
Modules + glue code
=
rapid/easy application development
AWESOME!...
@crichardson
54. ... BUT
Variable quality
Multiple incomplete/competing modules, e.g. MySQL
drivers without connection pooling!
Often abandoned
No notion of a Maven-style local repository/cache =
repeated downloads
...
@crichardson
57. Agenda
Overview of NodeJS
JavaScript: Warts and all
The Reactor pattern: an event-driven architecture
NodeJS: There is a module for that
Building a front-end server with NodeJS
@crichardson
58. So why care about
NodeJS?
Easy to write scalable network services
Easy to push events to the browser
Easy to get (small) stuff done
It has a role to play in modern
application architecture
@crichardson
59. Evolving from a monolithic
architecture....
WAR
StoreFrontUI
Product Info
Service
Recommendation
Service
Review
Service
Order
Service
@crichardson
60. ... to a micro-service architecture
product info application
Product Info
Service
recommendations application
Store front application
StoreFrontUI
Recommendation
Service
reviews application
Review
Service
orders application
Order
Service
@crichardson
62. ...Presentation layer evolution
Browser
View
Web application
Static
content
Controller
JSON-REST
Model
HTML 5/JavaScript
IOS/Android clients
Events
RESTful
Endpoints
Event
publisher
@crichardson
63. Directly connecting the front-end to the backend
Chatty API
View
REST
Product Info
service
REST
Recommendation
Service
AMQP
Controller
Review
service
Model
Traditional web
application
View
Controller
Model
Browser/Native App
Web unfriendly
protocols
@crichardson
64. NodeJS as an API gateway
Single entry point
Browser
View
Controller
NodeJS
Model
HTML 5 - JavaScript
Product Info
service
REST
Recommendation
Service
AMQP
Review
service
REST
proxy
Native App
View
API
Gateway
REST
Controller
Model
Event
publishing
Optimized Client
specific APIs
Protocol
translation
@crichardson
65. Serving static content with
the Express web framework
var express = require('express')
, http = require('http')
, app = express()
, server = http.createServer(app)
;
From public
sub directory
app.configure(function(){
...
app.use(express.static(__dirname + '/public'));
});
server.listen(8081);
@crichardson
68. Implementing coarsegrained mobile API
var express = require('express'),
...;
app.get('/productdetails/:productId', function (req, res) {
getProductDetails(req.params. productId).then(
function (productDetails) {
res.json(productDetails);
}
});
@crichardson
70. Socket.io server-side
var express = require('express')
, http = require('http')
, amqp = require(‘amqp’)
....;
server.listen(8081);
...
var amqpCon = amqp.createConnection(...);
Handle
socket.io
connection
io.sockets.on('connection', function (socket) {
function amqpMessageHandler(message, headers, deliveryInfo) {
Republish
var m = JSON.parse(message.data.toString());
socket.emit(‘tick’, m);
as socket.io
};
event
amqpCon.queue(“”, {},
function(queue) {
queue.bind(“myExchange”, “”);
queue.subscribe(amqpMessageHandler);
Subscribe to
});
});
AMQP queue
https://github.com/cer/nodejs-clock
@crichardson
71. Socket.io - client side
<html>
<body>
The event is <span data-bind="text: ticker"></span>
<script src="/socket.io/socket.io.js"></script>
<script src="/knockout-2.0.0.js"></script>
<script src="/clock.js"></script>
</body>
</html>
Bind to model
Connect to
socket.io
clock.js
var socket = io.connect(location.hostname);
function ClockModel() {
self.ticker = ko.observable(1);
socket.on('tick', function (data) {
self.ticker(data);
});
};
ko.applyBindings(new ClockModel());
Subscribe
to tick event
Update
model
@crichardson
72. NodeJS is also great for writing
backend micro-services
“Network elements”
Simply ‘route, filter and transform packets’
Have minimal business logic
@crichardson
73. NodeJS-powered home security
FTP Server
Log file
FTP Server
Upload directory
Upload2S3
S3
SQS Queue
UploadQueue
Processor
DynamoDB
@crichardson
74. Summary
JavaScript is a very flawed language
The asynchronous model is often unnecessary; very
constraining; and adds complexity
BUT despite those problems
Today, NodeJS is remarkably useful for building networkfocussed components
@crichardson