Wednesday, 27 April 2016

Preparing for CSS grids

I was just listening to "The Web Ahead" podcast where they were talking about the upcoming CSS grid system. If you can spare the time, go and take a listen (episode 114). Upcoming in the sense that there is a spec, but it's not yet mainstream - you can enable it through a flag in WebKit based browsers; use Firefox nightly (e.g I'm on Chrome 51 and it seems to be available). So you can't use it in your production applications just yet - soon'ish.

So, to get started (if using Chrome), you will want to enable experimental web platform features:



You can see the current working draft here - https://www.w3.org/TR/css-grid-1/

It's worth taking note of the motivation behind this. If you look for CSS layouts, you will see a bunch of solutions (probably involving floats) but using techniques not really designed for a full page layout.

Let's get started making a grid. The containing element should have a display property of either grid or inline-grid.

Without specifying any column widths, when placing elements on the same row, the column widths will be of equal widths.

As per:

<div id="gridContainer">
  <div id="col1">one</div>
  <div id="col2">two</div>
  <div id="col3">three</div>
</div>

<style type="text/css">
#gridContainer {
  display: grid;
}

#col1 {
  grid-column: 1;
  background-color:lightgreen;
}

#col2 {
  grid-column: 2;
  background-color:lightcyan;
}

#col3 {
  grid-column: 3;
  background-color:lightsalmon;
}
</style>


Output:



You may want span elements over multiple columns - in that case, imagine numbered lines for each column. Values are separated with a forward slash (/), and here you would specify the starting column  line and the ending column line. Alternatively, we can specify the number columns to span, prefixed with the span keyword

If we update the CSS to:

#gridContainer {
  display: grid;
}

#col1 {
  grid-row: 1;
  grid-column: 1 / 2;
  background-color:lightgreen;
}

#col2 {
  grid-row: 2;
  grid-column: 1 / 3;
  background-color:lightcyan;
}

#col3 {
  grid-row: 3;
  grid-column: 1 / 4;
  background-color:lightsalmon;
}

or

#gridContainer {
  display: grid;
}

#col1 {
  grid-row: 1;
  grid-column: 1 / span 1;
  background-color:lightgreen;
}

#col2 {
  grid-row: 2;
  grid-column: 1 / span 2;
  background-color:lightcyan;
}

#col3 {
  grid-row: 3;
  grid-column: 1 / span 3;
  background-color:lightsalmon;
}


We get:



We can make our styles more readable, by specifying the layout spec in the containing element, and giving each line a name. This is done in square parenthesis ([,]). In the container element, we define the layout spec with the property grid-template-columns and grid-template-rows. Here, you would specify the width / height of the columns / rows.

(note: If you don't declare the columns and rows, as per the previous example, the grid is implicitly created based on your grid data - an explicit layout makes the grid more concise)

So, with that applied, when specifying the column - we can use the name rather than the column index, as per:

#gridContainer {
  display: grid;
  grid-template-columns: [start] 100px [col2] 100px [col3] 100px [end];
  grid-template-rows: [top] auto [middle] auto [end];
  
}

#col1 {
  grid-row: top;
  grid-column: start / col2;
  background-color:lightgreen;
}

#col2 {
  grid-row: middle;
  grid-column: start / col3;
  background-color:lightcyan;
}

#col3 {
  grid-row: end;
  grid-column: start / end;
  background-color:lightsalmon;
}


If you rows and columns follow the same spec for the whole page, you can use the repeat function which accepts two parameters - 1. How many times to repeat; and 2. the column/row spec.


e.g.: grid-template-columns: repeat(4, [col] 100px)

In this example, when referring to the column we would use the index, or the name "col index"

Well, I just wanted to give a brief overview of this new technologies. There are countless examples on http://gridbyexample.com/ that you can check out/try out.

Thursday, 21 April 2016

Setting up and consuming your data securely with ORDS

Setting up

So, you have some data you want to expose to third parties with a REST API, but you want to do so in a secure manner. This is all possible with ORDS using the OAuth2 mechanisms built in. To keep things simple, I will use the sample data (emp, dept) REST functions - and this has all be done in a workspace named "company_reports". And the set up will be done from within APEX - there is an alternative approach of setting everything up using the ORDS API, a subject for another day.

So, go to SQL Workshop and RESTful Services. If you don't already have the module "oracle.example.hr", click on the "Reset Sample Data" link in the task list:




Now, what we want to do is secure this so that anybody with the URL can't just come in and access the data. In the same task list as above, go to the "RESTful Service Privileges" link and create a new privilege protecting the module "oracle.example.hr".



At this point, if you now go into your module and try to test access to the data, you will receive an error (401 Forbidden): "Access to this resource is protected. Please sign in to access this resource.".

Before moving onto 3rd party clients, you may want to actually access this data in your APEX application from within the same workspace. We first need to make sure the designated users have been assigned the privilege "RESTful Services".



To access the from the same workspace, in an authenticated APEX session, you need the header set: Apex-Session, with the value being the application id and session id, separated by a comma (e.g.: Apex-Session: 114,16530545902770). If this is not possible, you can pass a query string parameter _apex_session so the URL becomes like: http://example.com/ords/company_reports/hr/empinfo/?_apex_session=114,16530545902770.

Now that we have secured our REST API, how do we set it up so that it can be consumed in a third party application? You need to set up a client, but to be able to do that, you need a user with the role of "OAuth 2.0 Client Developer" (note: this role alone isn't enough to consume the data)





To register a client, you need to go the URL: /ords/workspace/ui/oauth2/clients/ - replacing workspace with your actual workspace name. So, in this example case: http://example.com/ords/company_reports/ui/oauth2/clients. Here you need to log in with the user you assigned the OAuth2 Client Developer role.

If this is your first time registering a client, you will be presented with an empty page with a button to register a client:



So, let's go ahead and set up a client. Click the Register Client button filling out all the fields. For this example, I'm going to go with the Code response type. After accepting or denying access to your data, you are re-directed to the URL specified in the Redirect URI field. This URL has some query string parameters, so in your consuming application you need a URL that will be able to interpret the query string parameters to respond to the received code. More on that in the consuming section.








After registering the client, go back into it to get the URL you need to request a token. In this case, it becomes:

http://example.com/ords/company_reports/oauth2/auth?response_type=code&client_id=XWRMqpgzfkQVejNtiGWc1Q..&client_secret=5GHYNCTCp4yaFPNMsxgr2A..&state=&_auth_=force

Consuming

So, now this is all set up, it's time to use it in our third party application. For this, I've made a workspace named "employee_consumer". The first thing we need to do here, is set up our callback URI as set up when registering the client. It's at this point we will need to get our access_code.

The two handler cases are:
  1. Success - returns access_code and state
  2. Error - returns error and state
So, set up a handler for the two cases


Success


The source will be PL/SQL code with the following logic (this example has client id and secrets hard coded - ideally, you will encapsulate these better):

declare 
    l_access_token_resp CLOB;
    l_resource_resp CLOB;
    l_success_vals apex_json.t_values;
    
    l_access_token varchar2(50);
    l_error varchar2(50);
    
    l_resource_url varchar2(200);
    
    procedure print(p_in varchar2)
    as
    begin
        htp.p(p_in || '<br />');
    end print;
    
begin

    l_resource_url := 'http://192.168.56.101:8080/ords/company_reports/hr/employees/';

    apex_web_service.g_request_headers(1).name := 'Content-Type';
    apex_web_service.g_request_headers(1).value := 'application/x-www-form-urlencoded';

    l_access_token_resp :=
        apex_web_Service.make_rest_request(
            p_url => 'http://192.168.56.101:8080/ords/company_reports/oauth2/token'
            , p_http_method => 'POST'
            , p_username => 'cnnD6yQud4G11bM7XtUYog..'--client id
            , p_password => '6tfLKgRat7At3OM2BRhqCQ..'--client secret
            , p_parm_name => apex_util.string_to_table('grant_type:code')
            , p_parm_Value => apex_util.string_to_table('authorization_code:' || :code)
        );
        
    apex_json.parse(
        p_values => l_success_vals,
        p_source => l_access_token_resp
    );
    
    l_access_token :=
        apex_json.get_varchar2 (
            p_values => l_success_vals,
            p_path   => 'access_token'
        );
        
    l_error :=
        apex_json.get_varchar2 (
            p_values => l_success_vals,
            p_path   => 'error'
        );
    
    print('Access token: ' || l_access_token);    
    
    print('Error: ' || l_error);
    print('*************************');
    print('Requesting: ' || l_resource_url);
    print( ' ');
    
    apex_web_service.g_request_headers(1).name := 'Authorization';
    apex_web_service.g_request_headers(1).value := 'Bearer ' || l_access_token;
    
    l_resource_resp :=
        apex_web_service.make_rest_request(
            p_url => l_resource_url
          , p_http_method => 'GET'  
        );
        
    htp.p(l_resource_resp);
end;



Error


begin

htp.p('Error: ' || :error);
htp.p('State: ' || :state);

end;


So, now we can test all this out. In our application, we add a button with the action to redirect to URL. We specify the URL as that what was given to is during the set up of the OAuth client.

http://example.com/ords/company_reports/oauth2/auth?response_type=code&client_id=cnnD6yQud4G11bM7XtUYog..&client_secret=6tfLKgRat7At3OM2BRhqCQ..&state=&_auth_=force


When clicking on the button, I'll be taken to a login prompt


If I enter the credentials of someone without access to the data (the user needs the role of RESTful Services) over REST, I get taken to the redirect URI.

http://example.com/ords/employee_consumer/tokenReceive?error=access_denied&state=2816474104155

On the other hand, if I enter login of someone with valid credentials, I will be (initially) taken to a page to give access to the data.


So, when clicking Allow access, we get taken to the callback URI, with the code query string parameter.

http://example.com/ords/employee_consumer/tokenReceive?code=Jzf-w9kzmIN6r9aAfcl1EA..&state=2816474104155

This code string is then used to request an access token that will be used to access the secured resources. The access token is requested from /ords/company_reports/oauth2/token (e.g. http://example.com/ords/company_reports/oauth2/token)

The GET handler I set up earlier is just set up to add some debug information, and output the resource to the page.

(note: In these examples, on my workstation I set up example.com to point to 192.168.56.101 - which is why most examples use example.com, but the above output is using the actual servers IP address)

In your actual set up, you would probably want to store the access/refresh tokens somewhere and redirect back to APEX - hopefully this gives a good starting point.

Also, refer to ORDS docs giving an example: http://docs.oracle.com/cd/E56351_01/doc.30/e56293/img_gallery_ap.htm#AELIG90015



Monday, 11 April 2016

Understanding variable scope

In APEX, we have two primarily languages we would tend to work with:

  • PL/SQL
  • JavaScript
So, it's worth being aware of how variable scoping works in any program units you are developing.

If you don't already know it, JavaScript has function level scope, rather than block level scope. If you come from C-based language, and declare a variable inside a for loop, for instance, you would not expect that variable to live on outside of the loop.

This is not the behaviour of JavaScript, so let's give this a test to see:



When the variable i is declared, it is actually hoisted up to the to the top of the function. If you added a statement to the top of the function referencing i, i would have the value of undefined rather than a ReferenceError about using an undeclared variable.

No matter where a variable is declared in JavaScript, it is hoisted to the top of the containing function - something to be aware of. That's why you will often see JavaScript programs with all variables declared at the top of the function - and considered good practice.

If we introduced this into a language such as Java, we would get a compilation error for trying to use an unknown symbol - that is of course because the variable is only available in that particular block.

So, then, how does PL/SQL behave?

Well, for starters, PL/SQL has a bit more structure to it requiring variables to be declared in the declaration block - as opposed to having the ability to declare anywhere throughout the body of the program (aside from loops where the iterator can be declared inline - `for i in 1..100`).

In saying that, you can declare more variables inline by nesting additional blocks, and those nested blocks will naturally inherit properties declared above them. 



The same applies to named sub-units.



If we add a variable to a sub-unit of the same name, then we have a new variable to work with without over-writing the existing variable. 


What may be useful, is that we are able access the variable from the parent program unit by prefixing the name or label of the program unit with dot notation when accessing the variable. Be careful here though, if you have a label with the same name as a named program unit, it will use the closest match.



Sunday, 3 April 2016

The making of my APEX competition dashboard map

The other day, I submitted my entry into the APEX dashboard competition. It was interesting, as I had never done any projects with map visualisations so gave me the opportunity to learn a little on the topic - now that I've submitted my entry and my demo is set up, I think it's time to share what I learnt along the way.

First of all, GovHack (Australia) has this article on all things maps - http://govhack-toolkit.readthedocs.org/technical/making-maps/. So, having read that, I decided D3JS was the way forward. I managed to find a sample of a German map set up using this library (D3JS and topoJSON) - http://bl.ocks.org/oscar6echo/4423770. It uses a JSON file that contains all the data points to render all the data, but I had no clue how this data was obtained/generated just from that example - so I kept digging.

Which led me onto this great article, which pretty much takes you step by step on drawing the map components: https://bost.ocks.org/mike/map/ - and importantly it tells you a place to get the data, and make it the the correct format (JSON) that D3JS can use. This resource is Natural Earth which has a great many collection of geographic data - http://www.naturalearthdata.com.

The conversion process involves two tools:

  1. ogr2ogr - generating a GeoJSON file
  2. topojson - generating a topoJSON file
This guide seems to reference an OS X tool for getting the ogr2ogr tool, so I instead did a search in my package manager and found that tool to be a part of the gdal-bin package

$ apt-cache search ogr2ogr
gdal-bin - Geospatial Data Abstraction Library - Utility programs

So I installed that package, and installed topojson using npm as per the article.

Next, I went ahead and grabbed the data for the map I wanted to produce. I ended up grabbing the 1:10m scale, although in retrospect I need not have gone for such a highly detailed scale. Being only interested in states, I grabbed the "Admin 1 – States, Provinces" data - with the download link: http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/cultural/ne_10m_admin_1_states_provinces.zip

Back to the guide, it had these commands:

ogr2ogr \
  -f GeoJSON \
  -where "ADM0_A3 IN ('GBR', 'IRL')" \
  subunits.json \
  ne_10m_admin_0_map_subunits.shp

topojson \
  -o uk.json \
  --id-property SU_A3 \
  --properties name=NAME \
  -- \
  subunits.json \
  places.json

It was pretty straight forward to see what the inputs meant. On the ogr2ogr command
  • format as GeoJSON
  • Filter by some country codes
  • output file
  • input file
and topojson:

  • output file
  • set id property
  • set state name
  • pass input files
(the example actually uses to GeoJSON files merged into one, whereas I only went with the one - states)

All looked pretty clear, except it was obviously referencing fields in the shape file, and I wondered how I was supposed to know which fields to use - aside from of course following that guide.

A little bit of online research, and I found there was a package on Ubuntu that was able to read the data in a shape file - qgis

This package with two GUI programs:
  1. QGIS Desktop
  2. QGIS Browser
The latter being the one I needed to use. So I launched it and opened the shape file that I downloaded earlier (extracted from the zip - ne_10m_admin_1_states_provinces.shp). Scrolling through that file, I was able to find the "adm0_a3" field that was referenced in that file - as was name, but I couldn't see SU_A3. 



After a bit of analysis, I decided to use the field "adm1_code" as the id field, given me the following two commands to run:

ogr2ogr -f GeoJSON -where "ADM0_A3 = 'DEU'" states.json ne_10m_admin_1_states_provinces.shp

topojson -o de.json --id-property adm1_code --properties name=name -- states.json

Once all that was done, it was just a matter of prototyping the map. I started by doing this in a local file on my computer, before moving it into APEX and eventually a plugin in APEX. 

By default, the map is rendered quite small, so it needs to be scaled up to some figure. I just experimented a bit with that - and found applying a height to the svg element itself made it the right size for the screen. So my general code became:

var projection = d3.geo.mercator()
    .scale(500);

var path = d3.geo.path()
    .projection(projection);

var svg = d3.select("#germanMap")
    .attr("height", computedHeight);

d3.json(pluginFilePrefix + "de.json", function(error, de) {

    var states = topojson.feature(de, de.objects.states);

    svg.selectAll(".state")
        .data(states.features)
        .enter().append("path")
        .attr("class", function(d) { return d.id + " germanState"; })
        .attr("d", path)
        .on("click", germanMapRenderer.onClickState);
});

Here, I applied the adm1_code as a class to each state so I could apply the appropriate styles (for the purpose of this project, I wanted a heat map of the states based on population numbers) and also a class named germanState just to react on a click event on that class.

The full working example can be seen here: https://apex.oracle.com/pls/apex/f?p=94455
...and any code related to the project here: https://github.com/tschf/2016ADCEntry