Mais conteúdo relacionado Semelhante a Node.js and Oracle Database: New Development Techniques (20) Node.js and Oracle Database: New Development Techniques2. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Node.js and Oracle Database
Christopher Jones
Data Access Development
Oracle Database
22 May 2019
christopher.jones@oracle.com
@ghrd
New Development Techniques
3. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 5
About Me
Christopher Jones
Product Manger for Oracle Database,
Scripting Language Drivers (and related
technology)
Contact information
christopher.jones@oracle.com
@ghrd
blogs.oracle.com/opal
4. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, timing, and pricing of any
features or functionality described for Oracle’s products may change and remains at the
sole discretion of Oracle Corporation.
6
5. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
8
Introduction
It Starts and Ends With Connections
Getting Data Out of the Database
Putting Data Into the Database
JSON and Simple Oracle Document Access
What’s coming in node-oracledb 4
1
2
3
4
5
6
6. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
9
Introduction1
2
3
4
5
6
7. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Node.js
• Middle-tier JavaScript runtime
– Built on Chrome’s V8 JavaScript engine
– Lightweight and efficient
– Event-driven, non-blocking I/O model
• Allows one language, front-end and back-end
• npm package ecosystem
– World’s largest repo of open-source libraries
• Open Source
– Under Node.js Foundation
10
8. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
node-oracledb
• Node.js module for accessing Oracle Database
• Development is under Apache 2.0 license
– GitHub repository for source code
– Ongoing features and maintenance by Oracle
• Installable from npm registry
Users can contribute under the Oracle Contributor Agreement.
Thanks to all who have contributed code, documentation and ideas
11
9. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 12
Under the Covers
app.js
node-oracledb
ODPI-C
Oracle Call Interface
in Oracle Client
libraries
Oracle Net Provides:
• Connectivity, Failover, Encryption,
and more
• Configure with tnsnames.ora,
sqlnet.ora
Oracle Client Provides:
• Data Access, Performance, High
Availability, and more
• Configure with oraaccess.xml
Node.js
10. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
node-oracledb Features
• Async/Await, Promises, Callbacks and Streams
• SQL and PL/SQL Execution / Fetching of large result sets / REF CURSORs
• Binding using JavaScript objects or arrays / DML RETURNING / Batch Statement Execution
• Large Objects: CLOBs and BLOBs and Strings/Buffers or Streams
• Smart mapping between JavaScript and Oracle types with manual mapping also available
• Query results as JavaScript objects or arrays
• Connection Pooling (Hetero & Homogeneous) with Aliasing, Queuing, Caching, Tagging, Draining
• External Authentication / Privileged Connections / Proxy Connections / Password Changing
• Row Prefetching / Statement Caching / Client Result Caching
• End-to-End Tracing / Edition-Based Redefinition
• Continuous Query Notification
• Simple Oracle Document Access (SODA)
• Oracle Database High Availability Features and Oracle Net Features
13
11. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 14
node-oracledb Classes
Base class
• Get connections or create pools
• Set configuration parameters
Connection Pooling
• Dramatically increases performance
• Built-in pool cache for convenience
SQL and PL/SQL Execution
• Transaction support w/data type conversion
• Bind using JavaScript objects or arrays
Read-consistent, pageable cursor
• Used for large result sets
• Callbacks, Promises or Node.js streams
Large object support
• Stream large LOBs with this class
• Can fetch smaller LOBs as string/buffer
Oracledb
Pool
Connection
ResultSet
Lob
12. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Starting with node-oracledb
1. Install Node.js
2. Install Oracle Instant Client
3. Create package.json
4. Run npm install
Confidential – Oracle Internal/Restricted/Highly Restricted 16
13. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
node-oracledb fine print
• Prebuilt node-oracledb 3.1 binary modules exist for:
– Windows, Linux and macOS
– Node.js 6, 8, 10, 11
• Upcoming node-oracledb 4.0 will be for Node.js 8+
• You can also build node-oracledb from source
• LTS strategy
Confidential – Oracle Internal/Restricted/Highly Restricted 17
14. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
oracledb.getConnection(
{user:"hr", password:"welcome", connectString:"localhost/orclpdb1"},
function(err, connection) {
if (err) {
console.error(err.message);
return;
}
connection.execute(
`SELECT department_id, department_name
FROM departments
WHERE manager_id < :id`,
[110], { outFormat: oracledb.ARRAY },
function(err, result) {
if (err) {
console.error(err.message);
return;
}
console.log(result.rows);
connection.close(function(err) {
if (err) {
console.error(err.message);
}
});
});
});
18
node-oracledb – Node.js Callback Style
15. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 19
node-oracledb - Node.js 8’s Async/Await
async function getEmployee(empid) {
let conn;
try {
connection = await oracledb.getConnection(
{user: "hr", password: "welcome", connectString: "localhost/orclpdb1" });
const result = await connection.execute(
`SELECT department_id, department_name
FROM departments
WHERE manager_id < :id`, [empid] );
console.log(result.rows);
} catch (err) {
console.error(err);
} finally {
if (connection) {
try {
await connection.close();
} catch (err) {
console.error(err);
}
}
}
}
$ node select.js
[ [ 60, 'IT' ],
[ 90, 'Executive' ],
[ 100, 'Finance' ] ]
16. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pro Tip:
• Using a fixed Oracle time zone helps avoid machine and deployment
differences
process.env.ORA_SDTZ = 'UTC';
Confidential – Oracle Internal/Restricted/Highly Restricted 20
TIP
17. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
21
It Starts and Ends With Connections
1
2
3
4
5
6
18. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Standalone Connections
const oracledb = require("oracledb");
let dbConfig = {user:"hr", password:"***", connectString:"localhost/XE"};
let connection = await oracledb.getConnection(dbConfig);
// ... Use connection
await connection.close();
22
Node.js DB
19. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pooled Connections
// Application initialization
let dbConfig = {user:"hr", password:"***", connectString:"localhost/XE”,
poolMin:2, poolMax:4, poolIncrement:1, poolTimeout:60};
let pool = await oracledb.createPool(dbConfig);
// General runtime
let connection = await pool.getConnection();
// ... Use connection
await connection.close();
// Application termination
await pool.close();
23
Node.js DB
20. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 24
Node.js Architecture (not entirely accurate)
Timers
TCP/UDP
Evented
File I/O
DNS
User Code
Thread Pool
Async API CallsMain Thread
Event / Callback Queue
Callback
Functions
21. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Threads vs Connections
• oracledb.createPool({ . . .,
poolMin:0, poolMax:10, poolIncrement:1, poolTimeout:60 }, . . .
– 10 users access the app
• Result: slower than expected throughput. May see deadlocks
– In-use connections wait for DB responses so they hold a thread in use
– Node.js has maximum 4 worker threads by default (upper limit is 128)
• Solution: increase Node.js thread limit before Node.js threadpool starts:
export UV_THREADPOOL_SIZE=10
– May want more threads than connections, to do non-database work
25
Scenario 1: Want to Allow 10 Concurrent DB Users
22. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Threads vs Connections
• Scenario:
– UV_THREADPOOL_SIZE=4
– App opens one connection
– promise.all on 1 x long running SELECT and 3 x INSERTs
• Result
– Each execute() gets thread from worker thread pool
– But the connection can only do one thing
• So the query will block the inserts, blocking the threads from doing other work
• Solution
– Keep control in the JavaScript layer
26
Scenario 2: promise.all
23. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Threads vs Connections
• Scenario:
– Each iteration of promise.all gets its own connection
– Each connection used to insert some data
• Can the database even handle (lots of) multiple connections?
– You have to balance workload expectations with actual resources available
• Transactional consistency not possible
• Data load solution shown in later slides!
27
Scenario 3: Parallelizing Data Load
24. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pro Tip: Avoid Connection Storms on DB Server
• Set poolMin = poolMax:
oracledb.createPool(
{ user: "hr", password: "welcome", connectString: "localhost/XE”,
poolMin: 10, poolMax: 10, poolIncrement: 0, poolTimeout: 60 })
• Or use Oracle Net Listener rate limit settings
– CONNECTION_RATE_listener name
– RATE_LIMIT
28
Node.js DB
TIP
25. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pro Tips: Basic High Availability
• Configure operating system network TCP timeouts
• Configure Oracle Net options on the DB and/or node-oracledb side e.g.
– SQLNET.OUTBOUND_CONNECT_TIMEOUT, SQLNET.RECV_TIMEOUT,
SQLNET.SEND_TIMEOUT, EXPIRE_TIME,(ENABLE=BROKEN)etc.
• connection.break()
– may need: DISABLE_OOB=on if firewall blocks out-of-band breaks
• connection.callTimeout with 18c Oracle Client libraries
29
TIP
26. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Immediately Usable Connections
• Connections in pools can become unusable due to network dropouts etc
• Use 19, 18, or 12.2 Oracle Client libraries
– these have an always-on, network check during pool.getConnection()
• Tune createPool()’s pingInterval setting
– Round-trip ping for detecting session issues
Confidential – Oracle Internal/Restricted/Highly Restricted 31
c = p.getConnection()
c.execute() // error
c = p.getConnection()
c.execute() // error
X
X
27. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
poolPingInterval
poolPingInterval Behavior at pool.getConnection()
< 0 Never check for connection aliveness
= 0 Always check for each pool.getConnection()
> 0
(default is 60)
Checks aliveness with round-trip ping if the connection has been
idle in the pool (not "checked out" to the application by
getConnection()) for at least poolPingInterval seconds.
Confidential – Oracle Internal/Restricted/Highly Restricted 32
If a ping detects an unusable connection, it is dropped and a new one created before p.getConnection() returns
28. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pro Tips: Usable Connections
• Always do error checking and recovery after execute()
– network failures could occur anytime after getConnection()
– Could disable poolPingInterval to get ultimate scalability
• Avoid firewall timeouts and DBAs killing “idle” sessions
– node-oracledb ping features may mask these problems
• scalability impact
– Use AWR to check connection rate
Confidential – Oracle Internal/Restricted/Highly Restricted 33
TIP
29. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 34
Use the pool cache to access pool by name
Pro Tip: No need to pass a pool around
dbConfig = {user:"hr", password:"XXX", connectString:"localhost/XE”,
poolMin:2, poolMax:4, poolIncrement:1, alias:"mypool"};
/* pool = */ await oracledb.createPool(dbConfig);
connection = await oracledb.getConnection("mypool");
dbConfig = {user:"hr", password:"XXX", connectString:"localhost/XE”,
poolMin:2, poolMax:4, poolIncrement:1};
await oracledb.createPool(dbConfig);
connection = await oracledb.getConnection();
TIP
30. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Connection State
• Many applications set connection state
– e.g. using ALTER SESSION commands to set the date format, or a time zone
– For all practical purposes, a 'session' is the same as a 'connection’
• Pooled connections retain state after being released back to the pool
– Next user of the connection will see the same state
– But will the next user get the same connection or a brand new one?
Confidential – Oracle Internal/Restricted/Highly Restricted 35
Node.js DB
31. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pro Tip: Avoid unnecessary ALTER SESSIONs
connection = await oracledb.getConnection(dbConfig);
connection.execute(`ALTER SESSION . . . `);
Use a connection callback function to set session state
Confidential – Oracle Internal/Restricted/Highly Restricted 36
TIP
X
32. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Connection Pool sessionCallback
await oracledb.createPool(
{user: un, password: pw, connectString: cs, sessionCallback: initSession});
// Called only when the pool selects a brand new, never-before used connection.
// Called before pool.getConnection() returns.
function initSession(connection, requestedTag, cb) {
connection.execute(`ALTER SESSION SET TIME_ZONE = 'UTC'`, cb);
}
. . .
connection = await oracledb.getConnection(); // get connection in UTC
let result = await connection.execute(sql, binds, options);
await connection.close();
See examples/sessionfixup.js
Confidential – Oracle Internal/Restricted/Highly Restricted 37
Scenario 1: when all connections should have the same state
33. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Connection Pool sessionCallback
poolMax
Service
Invoked
getConnection()
calls
ALTER SESSION
calls
SELECT
calls
Total SQL
Calls
Without a callback 4 1000 1000 1000 1000 2000
Confidential – Oracle Internal/Restricted/Highly Restricted 38
Scenario 1: when all connections should have the same state
Micro-service where each service invocation does one query
poolMax
Service
Invoked
getConnection()
calls
ALTER SESSION
calls
SELECT
calls
Total SQL
Calls
Without a callback 4 1000 1000 1000 1000 2000
With a callback 4 1000 1000 4 1000 1004
34. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pro Tip: Connection Pool sessionCallback
Avoid round-trips
function initSession(connection, requestedTag, cb) {
connection.execute(
`begin
execute immediate
'alter session set nls_date_format = ''YYYY-MM-DD''
nls_language = AMERICAN';
execute immediate
' . . . '; -- other SQL
end;`,
cb);
}
Confidential – Oracle Internal/Restricted/Highly Restricted 39
TIP
35. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Connection Pool Tagging
Set a connection tag to represent connection state
connection = await pool.getConnection();
// Set state and update the tag
connection.execute(`ALTER SESSION . . . `, cb);
connection.tag = 'NAME1=VALUE1;NAME2=VALUE2’;
await connection.close();
Confidential – Oracle Internal/Restricted/Highly Restricted 40
Scenario 2: when connections need differing state
36. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Connection Pool Tagging
Requesting an already tagged pooled connection
conn = await oracledb.getConnection({poolAlias:'default', tag:'USER_TZ=UTC'});
When using a sessionCallback this will:
– Select an existing connection in the pool that has the tag USER_TZ=UTC
• sessionCallback is NOT called
– Or, select a new, previously unused connection in the pool (which will have no tag)
• sessionCallback is called
– Or, select a previously used connection with a different tag
• The existing session state and tag are cleared
• sessionCallback is called
Confidential – Oracle Internal/Restricted/Highly Restricted 41
Scenario 2: when connections need differing state
37. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Connection Pool Tagging
conn = await oracledb.getConnection({poolAlias:'default’,
tag:'USER_TZ=UTC', matchAnyTag: true});
• This will:
– Select an existing connection in the pool that has the requested tag
• sessionCallback is NOT called
– Or, select another connection in the pool
• Any existing connection state and tag are retained
• sessionCallback is called
Confidential – Oracle Internal/Restricted/Highly Restricted 42
Scenario 2: when connections need differing state
38. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Connection Pool Tagging
Confidential – Oracle Internal/Restricted/Highly Restricted 43
Scenario 2: when connections need differing state
function initSession(connection, requestedTag, cb) {
console.log(`requested tag: ${requestedTag}, actual tag: ${connection.tag}`);
// Parse requestedTag and connection.tag to decide what state to set
. . .
// Set the state
connection.execute(`ALTER SESSION SET . . .`,
(err) => {
connection.tag = requestedTag; // Update to match the new state
cb(err); // Call cb() last
});
}
See examples/sessiontagging1.js and examples/sessiontagging2.js
39. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
End Connections When No Longer Needed
• Releasing connections is:
– Required for pooled connections
– Best practice for standalone connections
• Don’t release connections until you are done with them, otherwise:
– “NJS-003: invalid connection”, “DPI-1010: not connected”, “ORA-12537:
TNS:connection closed”, etc.
– Example scenario: promise.all error handler releasing the connection
• Handler is called on first error
– Example scenario: streaming from a Lob instance
45
40. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Connection Pool Draining and Force Closing
• Closing the pool closes connections cleanly
• May need: DISABLE_OOB=on if firewall blocks out-of-band breaks
46
// close pool only if no connections in use
await pool.close();
// close pool when no connections are in use, or force it closed after 10 seconds
// No new connections can be created but existing connections can be used for 10 seconds
await pool.close(10);
// force the pool closed immediately
await pool.close(0);
41. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
47
Getting Data Out of the Database
1
2
3
4
5
6
42. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Query Methods
• Direct Fetches: execute("SELECT . . .", . . . )
– All rows are returned in one big memory-limited array, or limited to maxRows
• ResultSet : execute("SELECT . . .", . . ., { resultSet: true }, . . . )
– getRow(): Returns one row on each call until all rows are returned
– getRows(numRows): Returns batches of rows in each call until all rows are returned
• Stream: queryStream("SELECT . . .", . . . )
– Streams rows until all rows are returned
48
43. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Direct Fetches
• Default query behavior
– Easy to use
• Tune network transfer performance with fetchArraySize
– Memory can incrementally grow when the number of query rows is unknown, or
varies from execution to execution
– A single large chunk of memory doesn't need to be pre-allocated to handle the 'worst
case' of a large number of rows
• Drawbacks of direct fetches:
– One big array of results is needed
– Concatenation of record batches can use more memory than the final array requires
49
44. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 50
ResultSet Fetches
const result = await connection.execute(
`SELECT * FROM bigtable`,
[], // no bind variables
{ resultSet: true }
);
const rs = result.resultSet;
let row;
while ((row = await rs.getRow())) {
console.log(row);
}
await rs.close(); // always close the ResultSet
45. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pro Tips: Querying Data
• Use row-limiting SQL clauses:
SELECT last_name FROM employees ORDER BY last_name
OFFSET :offset ROWS FETCH NEXT :maxnumrows ROWS ONLY
• Use Direct Fetches for known small number of rows
– set fetchArraySize to number of rows, if known
– E.g for single row fetches set fetchArraySize to 1
• Use ResultSets or queryStream() for data sets of unknown or big size
– Always close ResultSets
• Cast date and timestamp bind variables in WHERE clauses:
– . . . WHERE cast(:ts as timestamp) < mytimestampcol
– . . . WHERE cast(:dt as date) < mydatecol
51
TIP
46. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Fetching LOBs
• Fetch and bind CLOB and BLOB as String and Buffer for performance
– Only use Lob Stream class for huge LOBs or if memory is limited
const result = await connection.execute(
`SELECT c FROM mylobs WHERE id = :idbv`,
[1],
{ fetchInfo: {"C": {type: oracledb.STRING}} }
);
const clob = result.rows[0][0];
console.log(clob);
52
47. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
53
Putting Data Into the Database
1
2
3
4
5
6
48. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Single Row DML is Easy
SQL> CREATE TABLE mytable (key NUMBER(9) NOT NULL, fruit VARCHAR2(40));
result = await connection.execute(
"INSERT INTO mytable VALUES (:k, :f)",
{ k: 1, f: 'apple' }, // Bind values
{ autoCommit: true }); // Or use connection.commit() later
console.log("Rows inserted: " + result.rowsAffected); // 1
54
49. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pro Tip: Use Batch Execution for DML
• Executes one DML statement (e.g INSERT, UPDATE) with many data values
• Reduces round trips: "A server round-trip is defined as the trip from the
client to the server and back to the client."
55
executeMany()
TIP
50. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 56
Basic Array DML
sql = "INSERT INTO mytable VALUES (:k, :f)";
data = [ { k: 1, f: "apple" }, { k: 2, f: "banana" } ];
options = { autoCommit: true,
bindDefs: {
k: { type: oracledb.NUMBER },
f: { type: oracledb.STRING, maxSize: 25 } } };
result = await connection.executeMany(sql, data, options);
console.log(result);
{ rowsAffected: 2 }
51. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 57
data = [ { k: 1, f: "apple" }, { k: 2, f: "banana" },
{ k: null, f: "cherry" }, { k: 3, f: "pear" },
{ k: null, f: "damson" } ];
options = { batchErrors: true,
bindDefs: {
k: { type: oracledb.NUMBER },
f: { type: oracledb.STRING, maxSize: 25 } } };
result = await connection.executeMany(sql, data, options);
console.log(result);
BatchErrors
{ rowsAffected: 3,
batchErrors:
[ { Error: ORA-01400: cannot insert NULL into
("CJ"."MYTABLE"."KEY") errorNum: 1400, offset: 2 },
{ Error: ORA-01400: cannot insert NULL into
("CJ"."MYTABLE"."KEY") errorNum: 1400, offset: 4 } ] }
52. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Batch Errors
• Batch errors are in the result, not the error
– Some classes of error will always trigger real errors
• Any autoCommit will be ignored if there are DML errors
– Valid rows will have been inserted and can be explicitly committed
• Attribute rowsAffected shows 3 rows were inserted
• Array of errors, one for each problematic record
– The offset is the data array position (0-based) of the problem record
58
53. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
{rowsAffected: 7, dmlRowCounts: [ 5, 2 ]}
59
DML Row Counts
// assume 5 apples and 2 bananas exist in the table
sql = "DELETE FROM mytable WHERE fruit = :f";
data = [ { f: "apple" }, { f: " banana" } ];
options = { dmlRowCounts: true,
bindDefs: {
f: { type: oracledb.STRING, maxSize: 25 } } };
result = await connection.executeMany(sql, data, options);
console.log(result);
54. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 60
sql = "INSERT INTO tab VALUES (:1) RETURNING ROWID INTO :2";
binds = [ ["One"], ["Two"], ["Three"] ];
options = {
bindDefs: [
{ type: oracledb.STRING, maxSize: 5 },
{ type: oracledb.STRING, maxSize: 18, dir: oracledb.BIND_OUT }
]
};
result = await connection.executeMany(sql, data, options);
console.log(result.outBinds);
DMLRETURNING
[ [ [ 'AAAmWkAAMAAAAnWAAA' ] ],
[ [ 'AAAmWkAAMAAAAnWAAB' ] ],
[ [ 'AAAmWkAAMAAAAnWAAC' ] ] ]
55. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
executeMany() Benchmark
• Data: for (var i = 0; i < numRows; i++)
data.push([i, "Test " + i]);
• SQL: sql = "INSERT INTO mytable VALUES (:1, :2)";
• Multiple inserts:
for (var i = 0; i < numRows; i++) {
await connection.execute(sql, data[i],
{autoCommit: (i < numRows-1 ? false : true)}); }
• Single insert:
await connection.executeMany(sql, data, {autoCommit: true});
61
56. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
executeMany() Benchmark – Scenario 1
• 1 number,
1 short string
• Local DB
• YMMV
1 10 100 1000 10000 100000
execute() 10 38 265 2,323 23,914 227,727
executeMany() 16 9 20 16 39 361
62
57. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 63
executeMany() Benchmark – Scenario 2
• 1 x NUMBER,
• 3 x VARCHAR2(1000))
• Remote DB
• YMMV
1 10 100 1,000
execute() 57 510 5,032 50,949
executeMany() 57 155 368 2,537
58. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pro Tips: executeMany()
• For huge data sets may need to call executeMany() multiple times with
batches of records
– autoCommit: false for first batches
– autoCommit: true for the last batch
• Use SQL*Loader or Data Pump instead, where appropriate
– These were added to Instant Client 12.2
64
TIP
60. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Calling PL/SQL is easy
const result = await connection.execute(
`BEGIN
myproc(:i, :io, :o);
END;`,
{
i: 'Chris', // type found from the data. Default dir is BIND_IN
io: { val: 'Jones', dir: oracledb.BIND_INOUT },
o: { type: oracledb.NUMBER, dir: oracledb.BIND_OUT }
});
console.log(result.outBinds);
Confidential – Oracle Internal/Restricted/Highly Restricted 67
61. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 68
SQL> CREATE OR REPLACE PROCEDURE testproc
(p_in IN NUMBER, p_out OUT NUMBER) AS
BEGIN
p_out := p_in * 2;
END;
plsql = "BEGIN testproc(:1, :2); END;";
binds = [ [1], [2], [3] ];
options = {
bindDefs: [
{ type: oracledb.NUMBER },
{ type: oracledb.NUMBER, dir: oracledb.BIND_OUT }
] };
result = await connection.executeMany(sql, data, options);
console.log(result.outBinds);
CallingPL/SQL
[ [ 2 ], [ 4 ], [ 6 ] ]
62. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
PL/SQL Collection Associative Array (Index-by) Binds
TYPE numtype IS TABLE OF NUMBER INDEX BY BINARY_INTEGER;
PROCEDURE myinproc(p_id IN NUMBER, p_vals IN numtype) IS BEGIN
FORALL i IN INDICES OF p_vals
INSERT INTO sometable (id, numcol) VALUES (p_id, p_vals(i));
END;
await connection.execute(
"BEGIN myinproc(:idbv, :valbv); END;",
{ idbv: 1234,
valbv: { type: oracledb.NUMBER,
dir: oracledb.BIND_IN,
val: [1, 2, 23, 4, 10] } };
69
63. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
70
JSON and Simple Oracle Document Access
1
2
3
4
5
6
64. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
JSON with Oracle Database 12.1.0.2
SQL> CREATE TABLE myjsontab (
c CLOB CHECK (c IS JSON)) LOB (c) STORE AS (CACHE);
myContent = {name: "Sally", address: {city: "Melbourne"}};
json = JSON.stringify(myContent);
result = await connection.execute(
'insert into myjsontab (c) values (:cbv)',
{ cbv: json }
);
71
Inserting
65. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
JSON with Oracle Database 12.1.0.2
result = await connection.execute(
`select c from myjsontab t where t.c.name = :cbv`, // 12.2 syntax
{ cbv: 'Sally' },
{ fetchInfo: {"C": {type: oracledb.STRING } }});
js = JSON.parse(result.rows[0]);
console.log('Name is: ' + js.name);
console.log('City is: ' + js.address.city);
//sql = "select c FROM myjsontab where json_exists(c, '$.address.city')";
72
Fetching
66. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Simple Oracle Document Access (SODA)
• Set of NoSQL-style APIs to create and access documents
– Documents are often JSON
– Oracle maps SODA calls to managed tables, indexes and sequences
– Query-by-example access makes data operations easy
• SQL is not necessary
– But could be used for advanced analysis
• SODA APIs also exist in Python, PL/SQL, C and Java
73
node-oracledb support requires DB 18.3 and Oracle Client 18.5/19.3
67. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 74
node-oracledb SODA Classes
Base SODA class
• Get SODA database
• Manipulate SODA collections
Set of Documents
• Create and find documents
Operation Builder
• Filters and QBE
• Retrieve, replace or remove documents
Document Cursor
• Iterate over retrieved documents
SODA Document
• Document content
• Access as JSON, object or buffer
SodaDatabase
SodaCollection
SodaOperation
SodaDocumentCursor
SodaDocument
Oracledb
Pool
Connection
ResultSet
Lob
68. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 75
Creating Collections
Using SODA in node-oracledb
SQL> grant SODA_APP to cj;
conn = await oracledb.getConnection({user:"cj",....});
soda = await conn.getSodaDatabase();
collection = await soda.createCollection("mycollection");
69. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 76
Inserting Documents
Using SODA in node-oracledb
content = {name: "Anthony", address: {city: "Edmonton"}};
await collection.insertOne(content);
doc = await collection.insertOneAndGet(content);
console.log("Key is:", doc.key);
// JSON (or Object or Buffer)
doc2 = soda.createDocument('{"name":"Venkat","city":"Bengaluru"}');
await collection.insertOne(doc2);
70. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Using SODA in node-oracledb
• Recommendation is to turn on oracle.autocommit
– Like with SQL, avoid auto commit when inserting lots of documents
• SODA’s DDL-like operations occur in autonomous transactions
77
Commit Behavior
71. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 78
Operation Builder – find()
Using SODA in node-oracledb
doc = await collection.find().key(key).getOne(); // A SodaDocument
content = doc.getContent(); // A JavaScript object
console.log('Retrieved SODA document as an object:');
console.log(content);
content = doc.getContentAsString(); // A JSON string
console.log('Retrieved SODA document as a string:');
console.log(content);
72. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 79
Operation Builder – filter() Query-by-example (QBE)
Using SODA in node-oracledb
// Find all documents with city names starting with 'S'
console.log('Cities starting with S');
documents = await collection.find()
.filter({"address.city": {"$like": "S%"}})
.getDocuments();
for (let i = 0; i < documents.length; i++) {
content = documents[i].getContent();
console.log(' city is: ', content.address.city);
}
73. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
80
What’s coming in node-oracledb 4
1
2
3
4
5
6
74. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
What’s coming in node-oracledb 4.0
• Install (compile) current code snapshot with:
npm install oracle/node-oracledb#dev-4.0
• Already landed:
– N-NAPI code refactor improving Node.js version compatibility
– Advanced Queuing for “RAW” queues, ie. Node.js String and Buffers
– Implicit Results
– SODA bulk insert
• Being investigated (no promises!):
– SQL and PL/SQL object binding & queries
• When? Planned CY2019
Confidential – Oracle Internal/Restricted/Highly Restricted 81
75. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 82
AQ ‘RAW’ Queue Demo
Oracle Advanced Queuing
Dequeue:
queue = connection.queue('MYQUEUE');
msg = await queue.deqOne();
await connection.commit();
console.log(msg.payload.toString());
Enqueue:
queue = connection.queue('MYQUEUE');
messageString = 'This is my message';
await queue.enqOne(messageString);
await connection.commit();
BEGIN
dbms_aqadm.create_queue_table('DEMO_RAW_QUEUE_TAB', 'RAW');
dbms_aqadm.create_queue('MYQUEUE', 'DEMO_RAW_QUEUE_TAB');
dbms_aqadm.start_queue('MYQUEUE');
END;
/
76. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 83
Wrap up
77. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
node-oracledb
• Connection management is the basis for good scalability
• Utilize the best query method for the task, and tune it
• Use executeMany() to insert/update/delete multiple records efficiently
• Use SODA’s NoSQL-style data access to simplify applications
84
78. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Resources
• Mail: christopher.jones@oracle.com
• Twitter: @ghrd, @AnthonyTuininga, @dmcghan
• Blog: https://blogs.oracle.com/opal
• Office Hours: https://tinyurl.com/NodeHours
• Facebook: https://www.facebook.com/groups/oraclescripting/
• Home: https://oracle.github.io/node-oracledb/
• Code: https://github.com/oracle/node-oracledb
85