Saturday, 30 September 2017

ES6 with SQLcl / JDK9

In JDK 8, the Nashorn engine was still on ES5. With the latest release (JDK9), they have included optional support for ES6, which can be enabled by way of an argument. So, to test out this functionality I will be trying with a new variable/constant construct called `const`.

Before developing my script for SQLcl, I sometimes like to use jjs to verify everything. jjs is an interactive shell that invokes the Nashorn engine. So by defualt it uses ES5, so we can write a script like so:

jjs> var a = Math.floor(Math.random()*100) * 55
jjs> print (a)
990
jjs>

With JDK9, we can invoke this tool with ES6 support via an optional argument `--language` where we can specifiy either es5 or es6.

$ jjs --language=es6
jjs> const a = Math.floor(Math.random()*100) * 55
jjs> a = 2
<shell>:1 TypeError: Assignment to constant "a"
jjs> print (a);
2695
jjs>


The above snippet shows I can use the new `const` construct that is a part of the ES6 spec within the Nashorn engine. See this article for more information about some of what's included: https://www.oracle.com/corporate/features/nashorn-javascript-engine-jdk9.html

This is good and all, but the question is, how do we get this working in SQLcl?

Well, thankfully, when invoking our Java programs we can set nashorn orguments with a command: `-Dnashorn.orgs=.`.

If you take a look at the SQLcl bash script, you will see that if you set some JAVA_OPTS values, that will get passed along to the call to SQLcl. So, if you want to run ES6 (which, why wouldn't you!) in your SQLcl scripts, simply set that variable accordingly.

export JAVA_OPTS="-Dnashorn.args=--language=es6"

So, to verify it works, I set up a little test script using `const` instead of `var` (which isn't supported in ES5). The script looks like:

print ("Set variable `unchangeableValue` to 55");
const unchangeableValue = 55;
try {
    print ("Trying to change variable to 0");
    unchangeableValue = 0;
} catch (e) {
    print ("Exception");
    print (e);
}


First pass, we will run using ES5 (the default), and the engine picks up on a unrecognized key word (const) and throws an exception:

$ sql /nolog

SQLcl: Release 17.3.0 Production on Sat Sep 30 16:27:27 2017

Copyright (c) 1982, 2017, Oracle.  All rights reserved.


SQL> script test.js
javax.script.ScriptException: :2:0 Expected an operand but found const
const unchangeableValue = 55;
^ in  at line number 2 at column number 0


Second pass, we set the JAVA_OPTS as specified above. The output then becomes:

$ JAVA_OPTS="-Dnashorn.args=--language=es6" sql /nolog

SQLcl: Release 17.3.0 Production on Sat Sep 30 16:29:17 2017

Copyright (c) 1982, 2017, Oracle.  All rights reserved.

SQL> script test.js
Set variable `unchangeableValue` to 55
Trying to change variable to 0
Exception
TypeError: Assignment to constant "unchangeableValue"
SQL>


And the script runs as we expect!

Tuesday, 19 September 2017

Setting up Oracle XE in Docker - Lessons Learned

For a little side project I'm working on, I had the need to set up a Docker instance of Oracle. I have read about Docker a while back and understand the general concept - but up until now, I hadn't had much experience with it (other that running the basic hello-world Docker example).

So, since I'm about using the free version of the Oracle Database at the moment (Oracle XE), I wanted specifically to build that in a VM. Now, I know that Oracle has a GitHub repository with some sample Docker files - unfortunately, I didn't have success with the Oracle XE version, so it wasn't a bad time to go and build my own.

Here are some things I learnt:

1. The default storage location on my Ubuntu system is: /var/lib/docker

This is a problem as I'm developing, since the way my disk is set up, the root partition soon fills up.

2. You can change the location that images are stored, by creating a json file: /etc/docker/daemon.json.

In here, you will want an entry "data-root" with the path to your desired location. The Docker daemon runs as a service, which is effectively the `dockerd` binary. So, whilst testing your config, it's a good idea to stop that (systemctl stop docker). Then, update your config and run `dockerd`. There is also a `debug` property (true|false) that you can set so you get additional output. The first time I ran the daemon with the config file in place, my system had to set up a storage pool

DEBU[0001] devmapper: Pool doesn't exist. Creating it.

This seemed to bring my system to a hault, but leaving my system and coming back it was doing something - so keep in mind that isn't a quick process!

Something curious is that this config file allows you also to specify the storage driver. Now, I don't know much about the Docker storage drivers at this stage, but without the config file it uses the `aufs` storage driver, however if I try to set that in my config file, I get an error about the driver not being available. So it's possible I'm missing some other corresponding entry.

Once all looks good, switch back to using the daemon as a service: systemctl start docker

3. The CMD argument in your Dockerfile is best as a single executable

The CMD argument is what get executed whenever you run a new instance of your image. For the image I built, my command was to start the oracle-xe service. So I started with CMD ["/etc/init.d/oracle-xe", "start"]. This is good, but the problem is when running in detached mode, because the CMD completes, the container also exits.

After doing a bit of research and looking at the Oracle's example Dockerfile, it is a good idea to watch a file - the Oracle one was monitoring an log file, but I so another example online suggested just to watch /dev/null. So effectively I needed to run two commands:

1. oracle-xe start
2. tail -f /dev/null

When I tried chaining these in the CMD property of the Dockerfile, the container threw errors.

Again, after some more research and looking at the example on Oracle's repository, if you have more than one program to run, you should place this in a simple shell script, and pass that into the CMD property.

So, I end up with a script:

#!/bin/bash
/etc/init.d/oracle-xe start
tail -f /dev/null

Then the Dockerfile referencing:

CMD ["/root/install/runXe.sh"]

4. The images are generally very minimal and need to have context set up

If you run the Ubuntu image:

docker run -it ubuntu:16.04

You will see there is very little environment variables set up:

PWD=/
SHLVL=1
HOME=/root
_=/usr/bin/env
root@b7535b34011e:/# pwd
/


So, it's not a bad idea to set up some basic environment such as the user

USER root
WORKDIR /root
ENV HOME /root 
ENV USER root


When copying files over, at least now we are placing them within the users home (/root) rather than the root of the file system.

5. It's a good idea to run interactively whilst developing

Docker supports two methods to run an image

-i, --interactive=true|false
    Keep STDIN open even if not attached. The default is false.

-d, --detach=true|false
    Detached mode: run the container in the background and print the new container ID. The default is false.

I like to run interactively most of the time to see everything is behaving as expected and to see any output. So the full command would be something like:

docker run -it <image_tag>

Replace i with d when all looks good!

If your CMD property isn't giving the desired result, you can override what is set in the image by passing the --entrypoint command line argument with the value of /bin/bash

docker run -it --entrypoint=/bin/bash <image_tag>

6. You need to set the shared memory size in the command line argument

XE requires at least 1gb of shared memory, so I am a generous guy and give 2gb. This needs to be set when running the image through an argument. So the previous example becomes:

docker run --shm-size=2g -it --entrypoint=/bin/bash <image_tag>

7. Avoid anything that depends on a hostname

When installing XE, the generated listener.ora will reference the hostname. Since each time you run an image in a new container the hostname will be something different, it's a good idea to update this so that you will be able to connect on each corresponding container.

See the example of the listener.ora from the Oracle repository here: https://github.com/oracle/docker-images/blob/master/OracleDatabase/dockerfiles/11.2.0.2/runOracle.sh#L98-L115

8. Finally, some useful commands

docker build -t   
docker run --shm-size=2g -it 

# Stop running containers
docker stop $(docker ps -q -a)
# Remove all containers
docker rm $(docker ps -a -q)
# Delete images with no tag/name
docker rmi $(docker images -f "dangling=true" -q)
# Delete all images
docker rmi $(docker images -q)