Sunday, 21 May 2017

SQLcl scripting and npm modules

Node, or rather npm, has a huge collection of submitted modules! And if you've been using SQLcl to connect with the database, you should know it has scriptable ability where you can for example run JavaScript code and Java APIs. So the question is, how do we get the script to pick up node modules?

During my research, I found a few suggested methods:
  1. J2V8
  2. Project Avatar
  3. jvm-npm
I didn't try the first two methods, but have had success using the third (jvm-npm). This is a library with the source code hosted over on GitHub -

At this point, it's worth noting - not everything will work, as it does not include the full NodeJS API. 

So, first step to do is you'll probably want to host that library somewhere in your project - in my example, which I will provide the link to at the end of the article, I place it at ./lib/jvm-npm.js. Worth mentioning, you can also source this over http in your script, but probably best to keep a local copy.

For this example project, I'm going to focus on the lodash node module. So, the next step will be to set up a package.json file where you list any node dependencies - this is purely so I can run `npm install` to install any dependencies my script requires. So go ahead and run `npm init` in your project directory. After completing the wizard add your dependencies. You should end up with something like:

  "name": "sqlcl-npm-demo",
  "version": "1.0.0",
  "description": "Example of using node modules in an SQLcl script",
  "main": "lodashExample.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  "author": "Trent Schafer",
  "license": "Apache",
  "dependencies": {
    "lodash": "^4.17.4"

Now, after that is done, run: `npm install`.

trent@birroth:~/Projects/sqlcl-npm-demo$ npm install
sqlcl-npm-demo@1.0.0 /home/trent/Projects/sqlcl-npm-demo
└── lodash@4.17.4
trent@birroth:~/Projects/sqlcl-npm-demo$ ls node_modules/

Ok, so we have all of our dependencies installed. Now, how to we get this to work in our SQLcl script?

In a typical node app, you would have something like:

var _ = require('lodash');

If you try this out now, you will get an error like so:

SQL> script lodashExample
javax.script.ScriptException: ReferenceError: "require" is not defined in  at line number 1
<stack trace trimmed>

So, in order to get `require` available to us, we need to make it so our script has this jvm-npm library loaded - that we place in our project earlier. If you're not aware, you can load other scripts into a nashorn script with the `load` function. Read more over at this article:

So, at the start our script, add the following call:


Then try re-running your script, and you should notice no errors this time around. So, it looks like we are good to go! Just to verify, let's take something out of the lodash API to see if all is working as expected. Lodash has an `nth` function that basically retrieves the specified index of the passed in array - with one little enhancement, it supports negative indexes to search backward - so let's try that out.

My full script becomes:

var _ = require('lodash');

var testArray = [

var secondLast = _.nth(testArray, -2);

print("The second last item was", secondLast);

Then, the test output:

trent@birroth:~/Projects/sqlcl-npm-demo$ sql vmtest/vmtest@//

SQLcl: Release 4.2.0 Production on Sun May 21 19:40:07 2017

Copyright (c) 1982, 2017, Oracle.  All rights reserved.

Connected to:
Oracle Database 11g Express Edition Release - 64bit Production

SQL> script lodashExample.js
The second last item was 164

Which all looks correct. Yay, success.

As I mentioned, it doesn't include the full node API, so you will probably need to be cautious going forward if you use this - but just offers that little bit more.

Working files from this example: