SIT Belgium 2019 - Debugging Node.js Applications in SCP CF

Troubleshooting Node.js applications
Debugging in SAP Cloud Platform Cloud Foundry
Vadim Klimov
SAP Inside Track Belgium
May 2019

Demo application
2
Houston, we have a problem!

Ideas on diagnostics and troubleshooting
3
Logging library (Winston, Log4js, Bunyan, Pino,
{logging library of your choice}) + HTTP logger
(Morgan, {HTTP logger of your choice})
console.log()
Simple, but may overfill source code with excessive
number of logging statements
More advanced, but may still overfill source code with
excessive number of logging statements
There shall be a right tool for the job. Mindful logging
– for continuous monitoring and diagnostics, verbose
tracing on demand – for common troubleshooting,
debugger – for advanced troubleshooting
Let’s go for a debugger!

Node.js debugging
Ahead of remote application debugging – debug in local environment, if possible
4
 Do not kick off remote debugging straightaway – firstly, assess feasibility of running a Node.js application
in a local environment in a representational way, reproducing an issue and debugging it locally.
 Reproduce integral components of an application environment (runtime, dependencies, etc.) in a local
environment, or ensure a local environment has access to corresponding remote dependencies, or mock
them.
 Use the same code base of the application when running it in local and remote environments.
 Few useful NPM packages to simplify access to application environment variables and unify it between
local and Cloud Foundry deployments:
 dotenv (https://www.npmjs.com/package/dotenv) – to load environment variables.
 cfenv (https://www.npmjs.com/package/cfenv) – to access an application container environment in
Cloud Foundry (when application runs in Cloud Foundry) or emulate it in a local environment (when
application runs locally).

Node.js debugger and debuggee
Simplified overview
5
Code editor
(Visual Studio Code, Atom, Brackets, …)
Node.js application
InspectorAPI
node --inspect[-brk][=port] {main module}
Browser built-in DevTools
(Chrome DevTools, …)
IDE
(WebStorm, Eclipse, Komodo, …)
 Arguments --debug and --debug-brk – to establish debugging session between debugger and
debuggee using V8 Debugging Protocol (legacy). Where possible, its successor – V8 Inspector Protocol –
shall be considered (can be enabled using arguments --inspect and --inspect-brk).
 --inspect-brk / --debug-brk – to pause debuggee before user code (invocation of main module).
 --inspect / --debug – to pause debuggee at active breakpoints.
Inspector
client
Inspector
client
Inspector
client

Developer’s machine SCP CF subaccount
Organization
Space
Node.js debugger and debuggee
Application deployed to Cloud Foundry
6
Container
Node.js application
InspectorAPI
CFCLI
Port forwarding
through SSH tunnel
for Inspector port
of Node.js application
Code editor
IDE
Inspector
client
Inspector
client
Inspector
client

Debugging Node.js application deployed to Cloud Foundry
Preparatory steps
7
 Step 1. Define a script to start
a Node.js application with an
enabled Inspector.
 Step 2. Use the defined script
as a start up command of the
deployed application.
 Step 3. Enable SSH access to
a container of a deployed
Node.js application.
 Step 4. Configure port
forwarding through SSH
tunnel for an Inspector port
of the deployed Node.js
application.
 Step 5. Create debugging
configuration to attach to a
local Node.js process
(representing local end of the
SSH tunnel).
Node.js application Cloud Foundry Debugger

Node.js application
Steps 1 and 2. Adjust application manifest files
8
 In Node.js application manifest
(package.json), define a script to start
the application with an enabled Inspector.
 In Cloud Foundry deployment manifest
(manifest.xml), use the defined script as
a start up command of the deployed
application.

Organization
Space
Cloud Foundry
Access to Inspector of Node.js application through SSH tunnel
9
Container
Check if SSH access is enabled at space level cf space-ssh-allowed {space}
Enable SSH access cf allow-space-ssh {space}
Disable SSH access cf disallow-space-ssh {space}
Check if SSH access is enabled at application level cf ssh-enabled {app}
Enable SSH access cf enable-ssh {app}
Disable SSH access cf disable-ssh {app}
* If SSH access has been disabled,
restart application after enabling SSH access cf restart {app}
Node.js application
InspectorAPI
CFCLI
cf ssh [-N] –L
[bind host:]{bind port}:
{host}:{port} {app}
Port forwarding through
SSH tunnel for Inspector port
of Node.js application
Code editor
IDE
Inspector
client
Inspector
client
Inspector
client

Cloud Foundry
Step 3. Enable SSH access to application container
10
 By default, SSH access is enabled at SCP Cloud Foundry space level, but it is not enabled at application
level.
 Command: cf enable-ssh {app}.
 Restart the application after enabling SSH access to its application container.
 SSH access has been enabled and an
application presents itself as SSH enabled,
but attempt to connect using SSH fails if
the application has not been restarted…
 …and attempt to connect using SSH after
the application has been restarted.

Cloud Foundry
Step 4. Configure port forwarding through SSH tunnel for Node Inspector port
11
 Command: cf ssh [-N] -L [bind host:]{bind port}:{host}:{port} {app}.
 Argument -N – to omit access to an SSH shell of an application container right after setting up port
forwarding. If the argument is provided, port forwarding through SSH tunnel is enabled and an SSH shell to
the application container can be started at a later time using command cf ssh {app}.
 In either case, port forwarding through SSH tunnel for a Node Inspector port is configured and usage of a
specified local port can be verified:
 When started, port forwarding through SSH tunnel does not verify if the remote is accessible, if the
remote port is opened and if the corresponding listener service is ready to accept requests on it.

SCP CF region – IaaS provider (AWS, GCP, Azure)
IP ranges: {check SAP and IaaS provider’s documentation}
Developer’s machine
Cloud Foundry
Access to Inspector of Node.js application through SSH tunnel – communication
12
Container
Node.js application
InspectorAPI
CFCLI
IDE
Code editor
Inspectorclient
node --inspect ./dist/app.js
cf ssh [-N] –L [bind host:]{bind port}:{host}:{port} {app}
cf ssh -N -L 9229:127.0.0.1:9229 weather-demo
Port of SSH proxy: 2222
Host of application within
its container: 127.0.0.1
Can be omitted when CF
CLI runs and is accessed
on local network interface
In sake of simplicity, default Inspector port (9229) is used and is mapped to the same bind port on a local
machine. A bind port can be any unused port and does not need to be the same as the forwarded port.
SSH daemon of
application container

Side note: Cloud Foundry
Access to service instance through SSH tunnel – overview
13
Organization
Space
Container
Node.js application
InspectorAPI
CFCLI
MongoDB client (example)
Service {…} client
IDE
Inspector
client
MongoDB
driver
Container
Service {…}
Container
MongoDB
(example)
Published port
Exposed portContainer IP
Host IP

Access to service instance through SSH tunnel – MongoDB example
14
 Option #1. Bind a service instance to an
application. Connection and authentication
information can be retrieved from an application
environment variable VCAP_SERVICES.
 Option #2. Create a service key for a service
instance. Connection and authentication
information can be retrieved from the service key.
 Option #3. For some services, connection and
authentication information can be retrieved from
service instance dashboard.
Restart the application before using port forwarding
for the service instance through SSH tunnel, if the
application was started before the service instance
creation.

Access to service instance through SSH tunnel – why is application restart required?
15
 Concept of application security groups in Cloud
Foundry.
 A new application security group is automatically
created for a newly created service instance at
space level.
 A new application security group does not affect
already started applications until they are
restarted.
 Accessibility of services exposed by a service
instance can be verified from an application
container.
 A new feature – dynamic egress policies (currently
in beta in Cloud Foundry): in contrast to
application security groups, does not require
application restart and can be set at application
level.

Cloud Foundry
Explore environment using SSH shell – application process and application directory
16
 Deployed and started Node.js application:
 Application directory – $HOME/app (/home/vcap/app) – contains pushed files of a Node.js application:

Debugger
Step 5. Debugging configuration
17
 request – “attach”, as debugger attaches to an
already running Node.js process (no need to launch it).
 port – local port which has been used when configuring
port forwarding through SSH tunnel.
 localRoot – Node.js project workspace on a local
machine (where debugger is executed).
 remoteRoot – directory that contains a Node.js
application on a remote server (in an application
container).
It is essential to specify both local root and remote root of Node.js project / application root directory to
allow correct source code mapping between debugger and debuggee.
Create debugging configuration (e.g. in launch.json in Visual Studio Code) to attach debugger to a locally
running Node.js process. Attach point is a local end of the SSH tunnel, through which debugger will reach
Inspector of a Node.js application deployed to Cloud Foundry:

Debugging Node.js application deployed to Cloud Foundry
Example
18

Cloud Foundry
Explore environment using Node REPL console – application process
19
 process.pid – process ID
 process.execPath – absolute path to
executable that started Node process.
 process.execArgv – Node.js specific
command line options passed when
starting Node process.
 process.mainModule – main module of
the application.
Enter Node REPL (Read-Eval-Print-Loop) console and explore process attributes – such as:

Debugging
Traditional breakpoint
20
Execution of a program flow is paused:
 for breakpoint –
whenever the program reaches corresponding
statement,
 for conditional breakpoint –
whenever the program reaches corresponding
statement and the specified condition is met
(evaluates to true).

Debugging
Logpoint
21
 In certain cases, a developer has no need
to pause program flow – instead, it is only
required to observe content (specific
variables) or evaluate expressions. In such
cases, use logpoints instead of traditional
breakpoints.
 Logpoint is a type of breakpoint that
produces the specified log entry and
outputs it to debug terminal without
suspending program execution. Essentially,
logpoint is a statement console.log()
that is added at the specific location of the
analyzed program at debug time, rather
than at development time.

Debugging
Logpoint – under the hood
22
Enable debug adapter trace to collect detailed information:
in debugging configuration, set value of an attribute
“trace” to:
 true – to output only to debug adapter log file, or
 “verbose” – to output both to debug adapter log file
and debug console.

Debugging
Hints and tips
23
 A lot of variables and properties can be declared when reaching certain processing steps, whereas a
developer might be interested in continuously observing only few of them. Use watch expressions to get
access to content of a selection of variables and to evaluate expressions:
 Use debug console as Node REPL console to execute statements.

Debugging
Hints and tips: transpiled Node.js application – TypeScript example
24
 For Node.js applications developed in programming languages that require transpiling (such as
TypeScript), ensure that generation of source map files is enabled (e.g. for TypeScript – in TypeScript
Compiler configuration) to make it possible to map originally developed source code and corresponding
JavaScript code that has been generated by compiler and is interpreted at runtime by JavaScript engine:
 Source map contains Base64 VLQ encoded mapping between original TypeScript and generated JavaScript
versions of source code.

Debugging Node.js application deployed to Cloud Foundry in production
25
Basic reflex of a developer when they
are asked to debug an application in a
production environment shall be…
Reminder: ALWAYS carefully assess possible risks and potential
impact on the debugged application and environment where it
runs, and evaluate consequences associated with running
debugging session in a production environment BEFORE you start
debugging. Any debugging exercise in a production environment
shall be taken with high caution.
…otherwise, imprudent debugging in a
production environment introduces a
(not so) small chance of a disaster.

26
Common case for a production environment:
 A Node.js application has been deployed to Cloud Foundry.
 The application has been started without an enabled Inspector.
 SSH access to the application container can be enabled, if needed.
 There is no point in configuring port forwarding for an Inspector port and attempting to connect to a
running application in debug mode in such a state, because Inspector does not listen on a dedicated port.

Enable Inspector in a running Node.js application
27
 Option #1. Make corresponding amendments to application manifest files and re-push the application to
Cloud Foundry.
Commonly is not acceptable in a production environment.
 Option #2. Connect to an application container using SSH and send signal USR1 to a process of the
application – this will enable Inspector for an already running Node.js application. Having this done,
configure port forwarding through SSH tunnel and proceed to remote debugging:

Cleanup after debugging – disable Inspector
28
Debugging session has been completed and debugger has been disconnected, but Inspector remains enabled
and listening for incoming connections. In contrast to enabling Inspector, it cannot be disabled by sending
signal to a process of the application (Inspector cannot be disabled from outside the process).
 Option #1. Restart an application – the application will be started with disabled Inspector.
Commonly is not acceptable in a production environment.
 Option #2. In Node REPL console, use Inspector API and trigger deactivation (closure) of Inspector:
require('inspector').close()
 Option #3. Enhance the application by implementing and registering signal event listener for a dedicated
process signal that is not in use yet. The listener shall implement similar logic as mentioned above and
disable Inspector if it is enabled. With this enhancement, Inspector can be disabled by sending
corresponding signal to a process of the application.

Debugging and challenge of the flow of time
29
Traditional debugger allows to step forward an analyzed program flow (forward debugging).
What if we need to rewind program flow back from the breakpoint (reverse debugging) to have a look into
what has happened within the program several statements ago?
 We need a time machine…
 … or Node.js runtime and debugger that support Time Travel Debugging (TTD).

Time travel debugging Node.js application
30
 Time travel debugging (reverse debugging) allows to step back to earlier executed statements of the
program. Can be also used for postmortem debugging – debugging of the program that does not run
anymore (crashed or has been terminated).
 Based on record-replay-snapshot architecture.
 Requires usage of an alternative JavaScript engine for Node.js runtime – Node ChakraCore
(https://github.com/nodejs/node-chakracore), and an enhanced JavaScript debugger – Jardis.
 As per the documentation, “this project is still work in progress and not an officially supported Node.js
branch”.
 The feature is not available in Chrome V8 JavaScript engine.
Warning: Stunts and tricks displayed hereafter are performed in
controlled environments, such as a local development
environment. Do not attempt to duplicate, re-create, or perform
the same or similar stunts and tricks in your production
environment.

Visual Studio Code extension
31
 An extension for Visual Studio Code – NodeChakra Time Travel Debug
(https://marketplace.visualstudio.com/items?itemName=ttd-trace-tools.node-chakracore-time-travel-
debugger):
 The extension enriches Visual Studio Code debugger with additional functionality to support time travel
debugging – such as reverse steps (reverse step local and reverse step dynamic operations) and replay:

Example
32

Constraints
33
If time travel debugging is so effective, why it is not enabled by default in debugging toolkit?
 Runtime overhead and costs associated with collection of large volume of non-deterministic data about
program state for each discrete snapshot during forward execution of the program.
 Delays when stepping through collected and persisted snapshots in reverse debugging and snapshots
replay modes.
The goal is to provide affordable time travel debugging system that addresses constraints and finds a
compromise between ultimate reverse debugging functionality and overhead that is incurred by a system.

Side note: Node Version Switcher
Rapid switch between JavaScript engines
34
 Node Version Switcher (https://github.com/jasongin/nvs) – to quickly switch between JavaScript engines
used by Node.js runtime, and Node.js versions.
 Debugging configuration to utilize NVS and specific Node.js engine:

Side note: Node Version Switcher (cont.)
Rapid switch between JavaScript engines
35
After switching Node.js engine, details of the used engine can be verified in Node REPL console by checking
process attributes – they will indicate which engine and version is currently used:

 Thank you
Dr. Vadim Klimov
SAP Integration Architect
SAP Community https://people.sap.com/Vadim.Klimov
LinkedIn https://www.linkedin.com/in/Vadim-Klimov

SIT Belgium 2019 - Debugging Node.js Applications in SCP CF

Recomendados

Recomendados

Mais conteúdo relacionado

Último

Último (20)

Destaque

Destaque (20)

SIT Belgium 2019 - Debugging Node.js Applications in SCP CF