Mais conteĂșdo relacionado Semelhante a EWD 3 Training Course Part 34: QEWD Resilient Mode (20) EWD 3 Training Course Part 34: QEWD Resilient Mode1. Copyright © 2016 M/Gateway Developments Ltd
EWD 3 Training Course
Part 34
QEWD Resilient Mode
Rob Tweed
Director, M/Gateway Developments Ltd
Twitter: @rtweed
2. Copyright © 2016 M/Gateway Developments Ltd
QEWD's Request Queue
âą QEWD uses the ewd-qoper8 module to provide
the master process queue and worker dispatch
mechanism
âą ewd-qoper8's queue is memory based
â A simple JavaScript array
âą If the master process stops unexpectedly
or whilst the system is very busy, you
might lose some request messages
3. Copyright © 2016 M/Gateway Developments Ltd
QEWD Audit Information
âą By default, QEWD does not keep any history or
audit information, eg:
â A history of incoming request messages
â A history of corresponding responses
4. Copyright © 2016 M/Gateway Developments Ltd
QEWD Resilient Mode
âą QEWD's built in, optional Resilient Mode
addresses these concerns, at least for
WebSocket/Ajax messages
â Backs up incoming requests to your embedded
database
âą Redis or Cache
âą Not yet available for GT.M
â Backs up response objects to your embedded
database
â Flags completed messages in the database
âą When your message handler's finished() function is invoked
5. Copyright © 2016 M/Gateway Developments Ltd
QEWD Recovery
âą If the QEWD master process stops and is
restarted
â Any clients/browsers that were previously running an
interactive application (WebSockets or Ajax) will re-
register automatically
âą Re-establishes the client's QEWD Session
â On re-registration, QEWD checks to see if that
Session had any uncompleted requests in the
Resilient Mode backup database
âą If so, they are re-queued automatically
6. Copyright © 2016 M/Gateway Developments Ltd
QEWD Resilient Mode Storage
âą The QEWD master process saves any incoming
messages asynchronously to your embedded
Global Storage database, using the Document
Store abstraction
â By default, saved in a document named ewdQueue
â You can opt to change this name
â This is done in parallel to ewd-qoper8's in-memory
queue
âą ewdQueue is only used as a backup
âą Resilient mode shouldn't significantly affect ewd-qoper8's
performance
7. Copyright © 2016 M/Gateway Developments Ltd
QEWD Audit Trail
âą The ewdQueue Document can be used as
an audit trail. It contains a record of
â Every incoming request object
â Every outgoing response object
âą You can determine the retention period
â Default is 1 hour
â Any records older than the retention period
are automatically purged by QEWD
8. Copyright © 2016 M/Gateway Developments Ltd
What about Web/REST Services?
âą Web/REST Service HTTP requests are not
currently saved into the ewdQueue Document
âą Reason:
â If the master process stops, the connection context
between the client and Express is lost
â So it would not be possible to return a response for a
re-queued HTTP request
âą This may change in future
â Particularly where you've linked the client to a QEWD
Session: this would allow re-establishment of client
connection context
9. Copyright © 2016 M/Gateway Developments Ltd
Enabling Resilient Mode
âą Simple change to your QEWD startup file
âą Add the resilientMode property to the
config object
âą Setting it to true applies the defaults:
â ewdQueue as the Document Name
â 1 hour retention period
âą Can over-ride these via an object instead
10. Copyright © 2016 M/Gateway Developments Ltd
Edit your QEWD Startup File
var config = {
managementPassword: 'keepThisSecret!',
serverName: 'My QEWD Server',
port: 8080,
poolSize: 2,
database: {
type: 'cache',
params: {
path: 'c:InterSystemsCache2015-2mgrâ
},
resilientMode: true
}
};
var qewd = require('qewd').master;
qewd.start(config);
Default mode
enabled
11. Copyright © 2016 M/Gateway Developments Ltd
Edit your QEWD Startup File
var config = {
managementPassword: 'keepThisSecret!',
serverName: 'My QEWD Server',
port: 8080,
poolSize: 2,
database: {
type: 'cache',
params: {
path: 'c:InterSystemsCache2015-2mgrâ
},
resilientMode: {
keepPeriod: 3600 * 24 // keep for 24 hours
}
}
};
var qewd = require('qewd').master;
qewd.start(config);
Specify the required
retention period
in seconds
12. Copyright © 2016 M/Gateway Developments Ltd
Edit your QEWD Startup File
var config = {
managementPassword: 'keepThisSecret!',
serverName: 'My QEWD Server',
port: 8080,
poolSize: 2,
database: {
type: 'cache',
params: {
path: 'c:InterSystemsCache2015-2mgrâ
},
resilientMode: {
queueDocumentName: 'myAuditTrail'
}
}
};
var qewd = require('qewd').master;
qewd.start(config);
Save the audit trail
information into a
Document named
myAuditTrail instead
13. Copyright © 2016 M/Gateway Developments Ltd
Edit your QEWD Startup File
var config = {
managementPassword: 'keepThisSecret!',
serverName: 'My QEWD Server',
port: 8080,
poolSize: 2,
database: {
type: 'cache',
params: {
path: 'c:InterSystemsCache2015-2mgrâ
},
resilientMode: {
queueDocumentName: 'myAuditTrail',
keepPeriod: 3600*24*30 // 30 day retention
}
}
};
var qewd = require('qewd').master;
qewd.start(config);
14. Copyright © 2016 M/Gateway Developments Ltd
Restart QEWD
âą Resilient Mode is now enabled
âą Logging / backup of queued requests and
responses will now commence
15. Copyright © 2016 M/Gateway Developments Ltd
Queue Document Structure
{
'message': {
{{timeStamp}}: {
'token': //EWD Session token,
'content': //queued request message payload
}
},
'pending': {
sessionToken: { {{timeStamp}}: ''}
}
}
Where {{timeStamp}} is a unique index/property created from
the system clock time
16. Copyright © 2016 M/Gateway Developments Ltd
Time-stamp format
Logic used is:
var time = process.hrtime();
var ix = time[0] * 1e9 + time[1]; // time in nanoseconds
var timeStamp = new Date().getTime() + '-' + ix;
So the date/time for each audit record can be obtained from:
timeStamp.split('-')[0]
17. Copyright © 2016 M/Gateway Developments Ltd
Queue Document Structure
During message processing by worker process,
workerStatus information added:
{
'message': {
{{timeStamp}}: {
'token': //EWD Session token,
'content': //queued request message payload,
workerStatus: 'started' || 'error'
}
},
'pending': {
sessionToken: { {{timeStamp}}: ''}
}
}
18. Copyright © 2016 M/Gateway Developments Ltd
Queue Document Structure
When processing by a worker process is
completed, workerStatus will be finished
and pending index record is deleted
{
'message': {
{{timeStamp}}: {
'token': //EWD Session token,
'content': //queued request message payload,
workerStatus: 'finished'
}
}
}
19. Copyright © 2016 M/Gateway Developments Ltd
Queue Document Structure
Response objects are logged in the
response array
{
'message': {
{{timeStamp}}: {
'token': //EWD Session token,
'content': //queued request message payload,
workerStatus: 'finished',
response: [
// one or more response objects
]
}
}
}
20. Copyright © 2016 M/Gateway Developments Ltd
QEWD Audit Trail
âą The ewdQueue Document is a standard
ewd-document-store Document, so it can
be accessed and analysed using the
standard DocumentStore / DocumentNode
APIs.
âą You can also inspect it using the qewd-
monitor application
âą It can be deleted or manipulated without
affecting QEWD's operation