JavaScript used to be confined to the browser. But these days, it becoming increasingly popular in server-side applications in the form of NodeJS. NodeJS provides event-driven, non-blocking I/O model that supposedly makes it easy to build scalable network application.
In this talk you will learn about the consequences of combining the event-driven programming model with a prototype-based, weakly typed, dynamic language. We will share our perspective as a server-side Java developer who wasn’t entirely happy about JavaScript in the browser, let alone on the server. You will learn how to use NodeJS effectively in modern, polyglot applications.
NodeJS: the good parts? A skeptic’s view (devnexus2014)
1. NodeJS: the good parts?
A skeptic’s view
Chris Richardson
Author of POJOs in Action
Founder of the original CloudFoundry.com
@crichardson
chris@chrisrichardson.net
http://plainoldobjects.com
@crichardson
2. Presentation goal
How a grumpy, gray-haired
server-side Java developer
discovered an appreciation
for NodeJS and JavaScript
@crichardson
8. Agenda
Overview of NodeJS
JavaScript: Warts and all
The Reactor pattern: an event-driven architecture
NodeJS: There is a module for that
Building a front-end server with NodeJS
@crichardson
15. Dynamic and weakly-typed
Dynamic:
Types are associated with values - not variables
Define new program elements at runtime
Weakly typed:
Leave out arguments to methods
Read non-existent object properties
Add new properties by simply setting them
@crichardson
17. JavaScript is a prototypal
language
Person
__proto__
sayHello
...
Prototype
function
...
inherited
Chris __proto__
name
nickname
“Chris”
“CER”
overrides
object specific
@crichardson
18. Prototypal code
Not defined
here
$ node
> var person = { sayHello: function () { console.log("Hello " + this.name); }};
[Function]
> var chris = Object.create(person, {name: {value: "Chris"}});
undefined
> var sarah = Object.create(person, {name: {value: "Sarah"}});
undefined
> chris.sayHello();
Hello Chris
create using
undefined
properties
> sarah.sayHello();
prototype
Hello Sarah
undefined
> chris.sayHello = function () { console.log("Hello mate: " + this.name); };
[Function]
> chris.sayHello();
Hello mate: Chris
undefined
@crichardson
19. JavaScript is Functional
function makeGenerator(nextFunction) {
Return a function
closure
var value = 0;
return function() {
var current = value;
value = nextFunction(value);
return current;
};
Pass function (literal)
as an argument
}
var inc = makeGenerator(function (x) {return x + 1; });
> inc()
0
> inc()
1
@crichardson
20. But JavaScript was created
in a hurry
http://thequickword.wordpress.com/2014/02/16/james-irys-history-of-programming-languages-illustrated-with-pictures-and-large-fonts/
@crichardson
21. Lots of flaws
The ‘Java...’ name creates expectations that it can’t satisfy
Fake classes: Hides prototypes BUT still seems weird
global namespace
scope of vars is confusing
Missing return statement = confusion
‘function’ is really verbose
‘this’ is dynamically scoped
Unexpected implicit conversions: 99 == “99”!
truthy and falsy values
52-bit ints
@crichardson
22. Dynamic + weakly-typed (+ event-driven)
code
+
misspelt property names
lots of time spent in the abyss
Essential: Use IDE integrated
with JSLint/JSHint + tests
23. Dynamic + weakly-typed code
Refactoring is more difficult
Understanding code/APIs is more difficult
24. Prototypal languages have
benefits BUT
Developers really like classes
JavaScript prototypes lack the powerful features from
the Self language
e.g. Multiple (and dynamic) inheritance
http://www.cs.ucsb.edu/~urs/oocsb/self/papers/papers.html
@crichardson
25. Verbose function syntax
> var numbers = [1,2,3,4,5]
> numbers.filter(function (n) { return n % 2 == 0; } ).map(function (n) { return n * n; })
[ 4, 16 ]
>
Versus
Prelude> let numbers = [1,2,3,4,5]
Prelude> map (n -> n * n) (filter (n -> mod n 2 == 0) numbers)
[4,16]
Or
scala> val numbers = 1..5
scala> numbers filter { _ % 2 == 0} map { n => n * n }
Vector(4, 16)
@crichardson
26. Verbose DSLs
describe('SomeEntity', function () {
beforeEach(function () { ... some initialization ... });
Jasmine
it('should do something', function () {
...
expect(someExpression).toBe(someValue);
});
});
Versus
class SomeScalaTest ...{
Scalatest
before { ... some initialization ... }
it should "do something" in {
...
someExpression should be(someValue)
}
@crichardson
27. JavaScript is the language
of the web
“You have to use the programming
language you have, not the one that you
might want”
@crichardson
28. It works but the result is
lost opportunities
and
impeded progress
@crichardson
29. But if you think that this isn’t
a problem then perhaps ....
“Stockholm syndrome ... is a
psychological phenomenon
in which hostages ... have
positive feelings toward their
captors, sometimes to the
point of defending them...”
http://en.wikipedia.org/wiki/Stockholm_syndrome
@crichardson
30. Martin Fowler once said:
"...I'm one of those who despairs that a
language with such deep flaws plays such an
important role in computation. Still the
consequence of this is that we must take
javascript seriously as a first-class language
and concentrate on how to limit the damage
its flaws cause. ...."
http://martinfowler.com/bliki/gotoAarhus2012.html
@crichardson
31. Use just the good parts
Douglas
Crockford
http://www.crockford.com/
@crichardson
32. Use a language that
compiles to JavaScript
TypeScript
Classes and interfaces (dynamic structural typing)
Typed parameters and fields
Dart
Class-based OO
Optional static typing
Bidirectional binding with DOM elements
Less backwards compatibility with JavaScript
Also has it’s own VM
@crichardson
33. CoffeeScript Hello World
Classes :-)
http = require('http')
Concise
class HttpRequestHandler
constructor: (@message) ->
Bound method
handle: (req, res) =>
res.writeHead(200, {'Content-Type': 'text/plain'})
res.end(@message + 'n')
handler = new HttpRequestHandler "Hi There from CoffeeScript"
server = http.createServer(handler.handle)
server.listen(1338, '127.0.0.1')
console.log('Server running at http://127.0.0.1:1338/')
@crichardson
36. About the Reactor pattern
Defined by Doug Schmidt in 1995
Pattern for writing scalable servers
Alternative to thread-per-connection model
Single threaded event loop dispatches events on
handles (e.g. sockets, file descriptors) to event handlers
@crichardson
38. Benefits
Separation of concerns - event handlers separated
from low-level mechanism
More efficient - no thread context switching
Simplified concurrency - single threaded = no
possibility of concurrent access to shared state
@crichardson
39. Drawbacks
Non-pre-emptive - handlers must not take a long time
Difficult to understand and debug:
Inverted flow of control
Can’t single step through code easily
Limited stack traces
No stack-based context, e.g. thread locals, exception handlers
How to enforce try {} finally {} behavior?
@crichardson
40. NodeJS app = layers of event
handlers
Recurring
events
from
Event
Emitters
Application code
Event
listener
HTTP
Callback
function
DB driver
...
One time
events:
async
operation
completion
Basic networking/file-system/etc.
NodeJS event loop
@crichardson
41. Async code = callback hell
Difficult to implement common scenarios:
Sequential: A
B
C
Scatter/Gather: A and B
C
Code quickly becomes very messy
@crichardson
42. Messy callback-based
scatter/gather code
The result of
getProductDetails
getProductDetails = (productId, callback) ->
productId = req.params.productId
result = {productId: productId}
Propagate error
makeCallbackFor = (key) ->
(error, x) ->
if error
callback(error)
else
result[key] = x
if (result.productInfo and result.recommendations and result.reviews)
callback(undefined, result)
Update result
Gather
getProductInfo(productId, makeCallbackFor('productInfo'))
getRecommendations(productId, makeCallbackFor('recommendations'))
getReviews(makeCallbackFor('reviews'))
Scatter
@crichardson
43. Simplifying code with
Promises (a.k.a. Futures)
Functions return a promise - no callback parameter
A promise represents an eventual outcome
Use a library of functions for transforming and
composing promises
Promises/A+ specification - http://promises-aplus.github.io/promises-spec
when.js (part of cujo.js by SpringSource) is a popular
implementation
Crockford’s RQ library is another option
@crichardson
45. Not bad but lacks Scala’s
syntactic sugar
class ProductDetailsService .... {
def getProductDetails(productId: Long) = {
for (((productInfo, recommendations), reviews) <getProductInfo(productId) zip
getProductInfo(productId) zip
getRecommendations(productId) zip
getRecommendations(productId) zip
getReviews(productId)
getReviews(productId))
yield
ProductDetails(productInfo, recommendations, reviews)
}
}
@crichardson
46. Long running computations
Long running computation
blocks event loop for
other requests
Need to run outside of main event loop
Options:
Community: web workers threads
Built-in: NodeJS child processes
@crichardson
47. Using child processes
parent.js
var child = require('child_process').fork('child.js');
function sayHelloToChild() {
child.send({hello: "child"});
}
Create child process
Send message to child
setTimeout(sayHelloToChild, 1000);
child.on('message', function(m) {
console.log('parent received:', m);
});
function kill() {
child.kill();
}
setTimeout(kill, 2000);
child.js
process.on('message', function (m) {
console.log("child received message=", m);
process.send({ihateyou: "you ruined my life"})
});
@crichardson
48. Modern multi-core machines
vs. single-threaded runtime
Many components of many applications
Don’t need the scalability of the Reactor pattern
Request-level thread-based parallelism works fine
There are other concurrency options
Actors, Software transactional memory, ...
Go goroutines, Erlang processes, ...
Imposing a single-threaded complexity tax on the
entire application is questionable
@crichardson
51. Thousands of community
developed modules
https://npmjs.org/
web frameworks, SQL/NoSQL
database drivers, messaging, utilities...
@crichardson
52. What’s a module?
foo.js
One or more JavaScript files
exports.sayHello = function () {
console.log(“Hello”);
}
Optional native code:
Compiled during installation
JavaScript != systems programming language
Package.json - metadata including dependencies
@crichardson
54. Easy to use
Core module OR
Path to file OR
module in node_modules
Module’s exports
var http = require(“http”)
var server = http.createServer...
@crichardson
56. There is a module for that...
Modules + glue code
=
rapid/easy application development
AWESOME!...
@crichardson
57. ... BUT
Variable quality
Multiple incomplete/competing modules, e.g. MySQL
drivers without connection pooling!
Often abandoned
No notion of a Maven-style local repository/cache: repeated
downloads for all transitive dependencies!
Modules sometimes use native code that must be compiled
...
@crichardson
60. Agenda
Overview of NodeJS
JavaScript: Warts and all
The Reactor pattern: an event-driven architecture
NodeJS: There is a module for that
Building a front-end server with NodeJS
@crichardson
61. So why care about
NodeJS?
Easy to write scalable network services
Easy to push events to the browser
Easy to get (small) stuff done
It has a role to play in modern
application architecture
@crichardson
62. Evolving from a monolithic
architecture....
WAR
StoreFrontUI
Product Info
Service
Recommendation
Service
Review
Service
Order
Service
@crichardson
63. ... to a micro-service architecture
product info application
Product Info
Service
recommendations application
Store front application
StoreFrontUI
Recommendation
Service
reviews application
Review
Service
orders application
Order
Service
@crichardson
65. ...Presentation layer evolution
Browser
View
Web application
Static
content
Controller
JSON-REST
Model
HTML 5/JavaScript
IOS/Android clients
Events
RESTful
Endpoints
Event
publisher
@crichardson
66. Directly connecting the front-end to the backend
Chatty API
View
REST
Product Info
service
REST
Recommendation
Service
AMQP
Controller
Review
service
Model
Traditional web
application
View
Controller
Model
Browser/Native App
Web unfriendly
protocols
@crichardson
67. NodeJS as an API gateway
Single entry point
Browser
View
Controller
NodeJS
Model
HTML 5 - JavaScript
Product Info
service
REST
Recommendation
Service
AMQP
Review
service
REST
proxy
Native App
View
API
Gateway
REST
Controller
Model
Event
publishing
Optimized Client
specific APIs
Protocol
translation
@crichardson
68. Alternatively: Server-side MVC with micro-web
apps
Small, cohesive, independently deployable web apps
Desktop
/mobile
Browser
Product Catalog
web app
Account mgmt
web app
Order mgmt
web app
REST
Product Info
service
REST
Recommendation
Service
AMQP
Review
service
Small footprint, nodeJS is ideal for micro-web apps
@crichardson
69. Serving static content with
the Express web framework
var express = require('express')
, http = require('http')
, app = express()
, server = http.createServer(app)
;
From public
sub directory
app.configure(function(){
...
app.use(express.static(__dirname + '/public'));
});
server.listen(8081);
@crichardson
73. Implementing coarsegrained mobile API
var express = require('express'),
...;
app.get('/productdetails/:productId', function (req, res) {
getProductDetails(req.params. productId).then(
function (productDetails) {
res.json(productDetails);
}
});
@crichardson
75. Socket.io server-side
var express = require('express')
, http = require('http')
, amqp = require(‘amqp’)
....;
server.listen(8081);
...
var amqpCon = amqp.createConnection(...);
Handle
socket.io
connection
io.sockets.on('connection', function (socket) {
function amqpMessageHandler(message, headers, deliveryInfo) {
Republish
var m = JSON.parse(message.data.toString());
socket.emit(‘tick’, m);
as socket.io
};
event
amqpCon.queue(“”, {},
function(queue) {
queue.bind(“myExchange”, “”);
queue.subscribe(amqpMessageHandler);
Subscribe to
});
});
AMQP queue
https://github.com/cer/nodejs-clock
@crichardson
76. Socket.io - client side
<html>
<body>
The event is <span data-bind="text: ticker"></span>
<script src="/socket.io/socket.io.js"></script>
<script src="/knockout-2.0.0.js"></script>
<script src="/clock.js"></script>
</body>
</html>
Bind to model
Connect to
socket.io server
clock.js
var socket = io.connect(location.hostname);
function ClockModel() {
self.ticker = ko.observable(1);
socket.on('tick', function (data) {
self.ticker(data);
});
};
ko.applyBindings(new ClockModel());
Subscribe
to tick event
Update
model
@crichardson
77. NodeJS is also great for writing
backend micro-services
“Network elements”
Simply ‘route, filter and transform packets’
Have minimal business logic
@crichardson
78. NodeJS-powered home security
FTP Server
Log file
FTP Server
Upload directory
Upload2S3
UploadQueue
Processor
S3
SQS Queue
IpCamViewer
Web App
DynamoDB
@crichardson
79. Summary
JavaScript is a very flawed language
The single-threaded, asynchronous model is often
unnecessary; very constraining; and adds complexity
BUT despite those problems
Today, NodeJS is remarkably useful for building networkfocussed components
@crichardson