Sunday, 3 April 2016

The making of my APEX competition dashboard map

The other day, I submitted my entry into the APEX dashboard competition. It was interesting, as I had never done any projects with map visualisations so gave me the opportunity to learn a little on the topic - now that I've submitted my entry and my demo is set up, I think it's time to share what I learnt along the way.

First of all, GovHack (Australia) has this article on all things maps - So, having read that, I decided D3JS was the way forward. I managed to find a sample of a German map set up using this library (D3JS and topoJSON) - It uses a JSON file that contains all the data points to render all the data, but I had no clue how this data was obtained/generated just from that example - so I kept digging.

Which led me onto this great article, which pretty much takes you step by step on drawing the map components: - and importantly it tells you a place to get the data, and make it the the correct format (JSON) that D3JS can use. This resource is Natural Earth which has a great many collection of geographic data -

The conversion process involves two tools:

  1. ogr2ogr - generating a GeoJSON file
  2. topojson - generating a topoJSON file
This guide seems to reference an OS X tool for getting the ogr2ogr tool, so I instead did a search in my package manager and found that tool to be a part of the gdal-bin package

$ apt-cache search ogr2ogr
gdal-bin - Geospatial Data Abstraction Library - Utility programs

So I installed that package, and installed topojson using npm as per the article.

Next, I went ahead and grabbed the data for the map I wanted to produce. I ended up grabbing the 1:10m scale, although in retrospect I need not have gone for such a highly detailed scale. Being only interested in states, I grabbed the "Admin 1 – States, Provinces" data - with the download link:

Back to the guide, it had these commands:

ogr2ogr \
  -f GeoJSON \
  -where "ADM0_A3 IN ('GBR', 'IRL')" \
  subunits.json \

topojson \
  -o uk.json \
  --id-property SU_A3 \
  --properties name=NAME \
  -- \
  subunits.json \

It was pretty straight forward to see what the inputs meant. On the ogr2ogr command
  • format as GeoJSON
  • Filter by some country codes
  • output file
  • input file
and topojson:

  • output file
  • set id property
  • set state name
  • pass input files
(the example actually uses to GeoJSON files merged into one, whereas I only went with the one - states)

All looked pretty clear, except it was obviously referencing fields in the shape file, and I wondered how I was supposed to know which fields to use - aside from of course following that guide.

A little bit of online research, and I found there was a package on Ubuntu that was able to read the data in a shape file - qgis

This package with two GUI programs:
  1. QGIS Desktop
  2. QGIS Browser
The latter being the one I needed to use. So I launched it and opened the shape file that I downloaded earlier (extracted from the zip - ne_10m_admin_1_states_provinces.shp). Scrolling through that file, I was able to find the "adm0_a3" field that was referenced in that file - as was name, but I couldn't see SU_A3. 

After a bit of analysis, I decided to use the field "adm1_code" as the id field, given me the following two commands to run:

ogr2ogr -f GeoJSON -where "ADM0_A3 = 'DEU'" states.json ne_10m_admin_1_states_provinces.shp

topojson -o de.json --id-property adm1_code --properties name=name -- states.json

Once all that was done, it was just a matter of prototyping the map. I started by doing this in a local file on my computer, before moving it into APEX and eventually a plugin in APEX. 

By default, the map is rendered quite small, so it needs to be scaled up to some figure. I just experimented a bit with that - and found applying a height to the svg element itself made it the right size for the screen. So my general code became:

var projection = d3.geo.mercator()

var path = d3.geo.path()

var svg ="#germanMap")
    .attr("height", computedHeight);

d3.json(pluginFilePrefix + "de.json", function(error, de) {

    var states = topojson.feature(de, de.objects.states);

        .attr("class", function(d) { return + " germanState"; })
        .attr("d", path)
        .on("click", germanMapRenderer.onClickState);

Here, I applied the adm1_code as a class to each state so I could apply the appropriate styles (for the purpose of this project, I wanted a heat map of the states based on population numbers) and also a class named germanState just to react on a click event on that class.

The full working example can be seen here:
...and any code related to the project here: