POI Data Provider - User and Programmers Guide

Version 5.4.a by Ari Okkonen, Adminotech Oy

Introduction

This document describes how to design and implement a distributed system utilizing the POI Data Provider Open Specification.

The available architecture gives wide possibilities to develop systems utilizing location data. Publicly available generic data can be augmented by business or application specific data. The system can utilize private data with restricted access.

Considerations in distributed system architecture are covered in System design considerations.

Server design principles are covered in Server programming guide.

Client design and implementation is covered in Client programming guide.

Background and Detail

This User and Programmers Guide relates to the POI Data Provider GE which is part of the Advanced Middleware and Web User Interfaces chapter. Please find more information about this Generic Enabler in the related Open Specification and Architecture Description. The Reference Implementation can be found at GitHub.

User guide

This section does not apply as this GE implementation is for programmers, who will invoke the APIs programmatically, there is no user interface as such.

Programmers guide

System design considerations

System architecture based on POI servers is quite free. It is possible to use publicly available and proprietary POI data providers together.

Using distributed data

A POI related application may have needs exceeding the capabilities of available public POI data providers, e.g.:

  • More data must be associated to POIs
  • More POIs are needed
  • Private extensions to POIs and data are needed
  • High-integrity data are needed

These needs can be addressed using several POI databases together. Private POI servers with possible access control can extend the scope of this technology to demanding critical solutions.

It is easy to combine data in several POI servers for use.

  • A client program can query several POI servers.
  • A POI server can query other POI servers.

A query may request

  • more POIs and
  • more data on given POIs.

Because POIs are identified by UUIDs it is possible to combine data about a POI from several otherwise unrelated POI data providers.

The UUID of a POI must be the same in different databases, or explicit mapping is needed.

Using separate data

Of course, public POI data providers are not required, if they are not useful for the application. The application can use POI data provider(s) and POIs totally separated from the publicly available ones.

Using SQL and NoSQL databases

SQL databases, including PostGIS, are good in searches with several limiting conditions. However, they are laborious to program for given data content. NoSQL databases are easy to use for arbitrary data, but they are worse in general searches. This POI architecture uses PostGIS database for searches based on spatial and other conditions. NoSQL database is used to store other information about POIs. PostGIS database provides the UUID of the POI. The UUID is used as a key to access a NoSQL database for the rest of the data.

Architecture examples

Following are some configurations combining several POI data providers.

Hierarchical POI server architecture
Business specific POI server uses another POI server

Wide POI server architecture
Client uses several POI servers

General POI server architecture
General case: A POI server combines data from several POI data providers

Interface reference

Interfacing between client and server is defined in POI Data Provider Open API Specification.

Security concerns

Internet servers are prone to all kinds of haphazard and malicious queries. In order to keep server data in useful condition, it is necessary to require enough secret data to authenticate the queries that change data in the server.

The POI DP API uses the auth_t parameter for the authorization token to positively identify the client. The reference implementation contains a full mechanism to manage user accounts and logins. You can use it or improve it according to your needs.

Authentication

NOTE: Since the release 5.3 user authentication is mandatory for add_poi, update_poi, and delete_poi requests. Authentication is optional for get_components, radial_search, bbox_search, and get_pois.

The reason is to diminish accidental and malicious garbling of POI data as well as support storing and viewing of confidential data.

User authentication is left to external service providers. User management and support of authentication services are implementation dependent.

An authentication token from an authentication service is used to get an authorization token from the POI data provider using the login request. The client uses the authorization token as the auth_t parameterin the subsequent requests to the POI data provider. The authorization token is invalidated using the logout request.

POI data providers configured to provide open_data assume view permission to everyone without authorization.

Server programming guide

Implementing a special-purpose POI Data Provider

You can use the provided POI Data Provider implementation as a basis for writing your own specific implementation. The part of the code you most likely have to modify the most are the database queries, as you may have specified your own database model. The following links contain guides for using both PostGIS and MongoDB:

Introduction to PostGIS

The MongoDB 2.4 Manual

Composing a POI Data Provider from multiple backends

You can write a POI Data Provider backend that composes data from multiple different POI Data Provider backends. The high-level application logic for such a implementation is presented here:

Handling spatial queries (e.g. radial_search)

  1. Forward the spatial query to a POI backend that is capable of handling them (core backend)
  2. Parse the response to the spatial query and create a list of POI UUIDs
  3. Loop through the rest of the POI backends and request additional data components for each UUID with get_pois request
  4. Construct the final output JSON by appending the additional data components for each POI

Handling get_pois queries

The get_pois query is a simpler case than the spatial queries, as you directly get the list of UUIDs as the request parameter: 1. Loop through the POI backends and request additional data components for each UUID with get_pois request 1. Construct the final output JSON by appending the additional data components for each POI

Client programming guide

See Interface reference for

  • details of communication with POI servers and
  • essential data structures provided by POI servers.

General on client design

General pattern of client operation is

  1. sorting out the area and other attributes of current interest - possible user interaction,
  2. generic query for POIs using spatial and other conditions,
  3. selecting the POI(s) to be focused on - possible user interaction,
  4. specific query for more data on specific POIs,
  5. using (showing) the acquired POI data - possible user interaction.

This manual describes how to program the queries to POI servers. Language used in examples is JavaScript due its wide availability in web browsers.

The common parts used in queries are described in separate section.

XMLHttpRequest is used to perform REST queries.

Obtaining the authorization token

The php/login_lib.js of the reference implementation contains programs and detailed instructions for login and logout operations to obtain and invalidate an authorization token auth_t.

login_lib.js
/*
A short JavaScript library to help utilizing the FIWARE POI Access Control
in application web pages.

NOTE: This library reserves several global names beginning LOGIN_, login_, 
      and logout_.

Usage
=====

1.  Copy following user interface elements and the library link to a proper
    location of your application page. You may have to edit the library
    link, if located separately from your application.
    ----
      <!-- Begin access control elements: buttons, name, and image -->
      <button id="login_b" style="" type="button" 
          onclick="login_click();"><b>Log In</b></button>
      <button id="logout_b" style="display:none" type="button" 
          onclick="logout_click();"><b>Log Out</b></button>
      <span id="login_user_name"></span>&nbsp;
      <img id="login_user_image" 
          src="data:image/gif;base64,R0lGODlhAQABAAAAACwAAAAAAQABAAA=" 
          width="32" height="32"><br>
      <!-- End access control elements -->
      <!-- Include login library -->
      <script type="text/javascript" src="login_lib.js"></script>
    ----

2.  Copy the following code template to the script part of your application. 
    Edit as needed. You may rename those my_logged_in and my_logged_out.
    ----
      // var LOGIN_SERVER = "http://www.example.org/poi_dp/"; // Note 
                                                         // trailing slash!
      var LOGIN_SERVER = ""; // can be left blank if in the same location

      var auth_t = ""; // to be used in subsequent requests
                                 // as the auth_t parameter
      var login_user_info = {}; // {name: string, image: url_string}

      var login_completed = my_logged_in; // called when login completed
      var logout_completed = my_logged_out; // called when logout completed

      function my_logged_in() {
        // Here comes your code that is executed on login
      }

      function my_logged_out() {  
        // Here comes your code that is executed on logout
      }

      // Ensure that the login button is enabled if the page is reloaded.
      document.getElementById("login_b").disabled = false;
    ----
*/

Querying available data formats

  • Full data schema supported by the server is available from http://<poi_server>/poi_dp/poi_schema.json e.g.:

    http://poi_dp.example.org/poi_dp/poi_schema.json

  • POI categories supported by the server are available from http://<poi_server>/poi_dp/poi_categories.json e.g.:

    http://poi_dp.example.org/poi_dp/poi_categories.json

Spatial query

Spatial query is used to find the POIs based on their location. See Interface reference for complete treatment of available query choices.

JavaScript skeleton for requesting POIs in given radius below gives an example how to implement a query in a client.

/*
  Query parameters:

  BACKEND_ADDRESS_POI - example: "http://poi_dp.example.org/poi_dp/"
  lat - latitude of the center point, degrees north
  lng - longitude of the center point, degrees east
  searchRadius - meters
  languages - a string array containing the accepted language codes
  auth_t - authorization token, if needed
*/
var query_url = BACKEND_ADDRESS_POI + "radial_search?" +
 "lat=" + lat + "&lon=" + lng + "&radius=" + searchRadius + "&component=fw_core" +
    ((auth_t != "") ?
    ("&auth_t=" + auth_t) : "");

poi_xhr = new XMLHttpRequest();
poi_xhr.onreadystatechange = function () {
  if(poi_xhr.readyState === 4) {
    if(poi_xhr.status  === 200) { 
      var resp_data = JSON.parse(poi_xhr.responseText);
      process_response(resp_data);
    }
    else { 
      console.log("failed: " + poi_xhr.responseText);
    }
  }
}

poi_xhr.onerror = function (e) {
  log("failed to get POIs");
};

poi_xhr.open("GET", query_url, true);
set_accept_languages(poi_xhr, languages);
poi_xhr.send();

Additional data query

Additional data for POIs already found can be requested using UUIDs of POIs as keys. Below is a JavaScript skeleton for requesting more data on given POIs.

/*
  Query parameters:

  BACKEND_ADDRESS_X_POI - example: "http://poi_dp.example.org/poi_dp/"
  uuids[] - string array containing UUIDs of interesting POIs
  languages - a string array containing the accepted language codes
  auth_t - authorization token, if needed
*/
var query_url = BACKEND_ADDRESS_X_POI + "get_pois?poi_id=" + join_strings(uuids, ",") +
    ((auth_t != "") ?
    ("&auth_t=" + auth_t) : "");

poi_xhr = new XMLHttpRequest();
poi_xhr.onreadystatechange = function () {
  if(poi_xhr.readyState === 4) {
    if(poi_xhr.status  === 200) { 
      var resp_data = JSON.parse(poi_xhr.responseText);
      process_response(resp_data);
    }
    else { 
      console.log("failed: " + poi_xhr.responseText);
    }
  }
}

poi_xhr.onerror = function (e) {
  log("failed to get POIs");
};

poi_xhr.open("GET", query_url, true);
set_accept_languages(poi_xhr, languages);
poi_xhr.send();

Handling received POI data

Query response consists of pois data on requested POIs, which consists of POI items having POI UUIDs as their keys.

Example data (shortened and annotated):

{
  "pois": {
    "6be4752b-fe6f-4c3a-98c1-13e5ccf01721": {
      "fw_core": {
        "categories": ["cafe"],
        "location": {<location of Aulakahvila>}, 
        "name": {
          "__": "Aulakahvila"
        },
        <more core data on Aulakahvila>;
      },
      <more data components on Aulakahvila>;
    }, 
    "ae01d34a-d0c1-4134-9107-71814b4805af": {<data on restaurant Julinia>},
    "1c022820-62dc-487b-95b4-6c344d6ba85e": {<data on library Tiedekirjasto Pegasus>},
    <more data on more POIs>
  }
}

First, the received text is parsed using JSON.parse() function.

NOTE: For security reasons do not use eval() function to parse the data! Maliciously formatted data can be used to compromise the security.

Then, the resulting data structure is processed for POI data. The function skeleton process_response below is an example how to process the data.

For the detailed structure of the response data see Interface reference.

process_response( data ) - skeleton
function process_response( data ) {

    var counter = 0, jsonData, poiData, pos, i, uuid, pois,
        contents, locations, location, searchPoint, poiCore,
        poiXxx;

    if (!(data && data.pois)) {
        return;
    }

    pois = data['pois'];

    /* process pois */

    for ( uuid in pois ) {
        poiData = pois[uuid];
        /*
           process the components of the POI
           e.g. fw_core component containing category, name,
           location etc.
           Taking local copies of the data can speed up later 
           processing.
        */
        poiCore = poiData.fw_core;
        if (poiCore) {
          /* fw_core data is used here */

        }
        /* Possible other components */
        poiXxx = poiData.xxx;
        if (poiXxx) {
           /* xxx data is used here */

        }
    }
}

Adding a new POI

Below is a JavaScript skeleton for adding a new POI to the POI-DP. For new POIs use poi_id = null.

Mirroring or storing auxiliary information for POIs: When you store copies or extra information of POIs to another server, set poi_id to the UUID of the POI. For auxiliary information databases you still need the fw_core components at least with fields location, name, and category.

BACKEND_ADDRESS_POI = "http://poi_dp.example.org/poi_dp/";
// auth_t - authorization token
// poi_id - optional - UUID of the new POI. Used e.g. when storing
//          extra data components to another server for a known POI.
//          null, if not used.

poi_data = {fw_core: {...},  -other components- };

function addPOI( poi_data ) {
    var restQueryURL;

    restQueryURL = BACKEND_ADDRESS_POI + "add_poi?auth_t=" + auth_t +
      (poi_id ? ("&poi_id=" + poi_id) : "");
    miwi_poi_xhr = new XMLHttpRequest();

    miwi_poi_xhr.overrideMimeType("application/json");

    miwi_poi_xhr.onreadystatechange = function () {
        if(miwi_poi_xhr.readyState === 4) {
            if(miwi_poi_xhr.status  === 200) {
              // React to successfull creation
              //  alert( "success: " + miwi_poi_xhr.responseText);
            }
            else {
              // React to failure
              // alert("failed: " + miwi_poi_xhr.readyState + " " + miwi_poi_xhr.responseText);
            }
        }
    }
    miwi_poi_xhr.onerror = function (e) {
        // React to error
        // alert("error" + JSON.stringify(e));
    };
    miwi_poi_xhr.open("POST", restQueryURL, true);
    miwi_poi_xhr.send(JSON.stringify(poi_data));
}

Updating POI

Below is a JavaScript skeleton for updating POI data in the POI-DP.

First, the data is fetched from the server.

BACKEND_ADDRESS_POI = "http://poi_dp.example.org/poi_dp/";
// auth_t - authorization token

var query_handle;

function POI_edit(uuid) {
/*
    uuid - the id of the POI to be updated
*/
    var restQueryURL, poi_data, poi_core;
    // get_for_update brings all language variants etc.
    restQueryURL = BACKEND_ADDRESS_POI + "get_pois?poi_id=" + uuid +
        "&get_for_update=true&auth_t=" + auth_t;

    console.log("3D restQueryURL: " + restQueryURL);
    query_handle = new XMLHttpRequest();

    query_handle.onreadystatechange = function () {
        if(query_handle.readyState === 4) {
            if(query_handle.status  === 200) { 
                //console.log( "succes: " + xhr.responseText);
                var json = JSON.parse(miwi_3d_xhr.responseText);
                var poi_edit_buffer = json.pois[uuid];

                /* Here a data editor is opened to edit the contents of poi_edit_buffer.
                   updatePOI is a callback that is called to send update to server.
                   uuid is passed to the callback. Other parameters are for information, only.
                */
                A_nonspecific_data_editor("update poi "  + uuid, "Edit POI data", poi_edit_buffer, uuid, updatePOI);

            }
        }
    }
    query_handle.onerror = function (e) {
        log("failed to get data");
    };

    query_handle.open("GET", restQueryURL, true);
    query_handle.send();

}

When editing is ready, the new version of data is sent to the server.

function updatePOI( poi_data, uuid ) {
/*
  poi_data is the updated version of POI data like:
    {
      "fw_core": {...},
      "fw_times": {...}
    }
  uuid is the id of the POI
*/
    var restQueryURL;
    var updating_data = {};
/* build updating structure like
    { 
      "30ddf703-59f5-4448-8918-0f625a7e1122": {
        "fw_core": {...},
        ...
      }
    }
*/
  updating_data[uuid] = poi_data;

  restQueryURL = BACKEND_ADDRESS_POI + "update_poi?auth_t=" + auth_t;
  query_handle = new XMLHttpRequest();

  query_handle.overrideMimeType("application/json");

  query_handle.onreadystatechange = function () {
      if(query_handle.readyState === 4) {
          if(query_handle.status  === 200) { 
            // Here we may notify the user of successfull update
            // alert( "success: " +query_handle.responseText);
          }
      }
  }
  query_handle.onerror = function (e) {
      // Something bad happened
      alert("error" + JSON.stringify(e));
  };
  // define the operation and URL
  query_handle.open("POST", restQueryURL, true);
  // send the data
  query_handle.send(JSON.stringify(updating_data));
}

Deleting POI

Below is a JavaScript skeleton for deleting a POI from the POI-DP.

BACKEND_ADDRESS_POI = "http://poi_dp.example.org/poi_dp/"; // for example

function POI_delete(uuid) {
  var restQueryURL, poi_data, poi_core;

  var cfm = confirm("Confirm to delete POI " + uuid);
  if (cfm)
  {
    // build the URL for delete
    restQueryURL = BACKEND_ADDRESS_POI + "delete_poi?poi_id=" + uuid + 
        "&auth_t=" + auth_t;

    miwi_3d_xhr = new XMLHttpRequest();
    // populate the request with event handlers
    miwi_3d_xhr.onreadystatechange = function () {
        if(miwi_3d_xhr.readyState === 4) {
            if(miwi_3d_xhr.status  === 200) { 
                // Notify user about success (if wanted)
                alert("Success: " + miwi_3d_xhr.responseText);
            }
        }
    }

    miwi_3d_xhr.onerror = function (e) {
        log("failed to delete POI " + JSON.stringify(e));
    };

    // Note: It seems to help DELETE if the client page is in the
    //       same server as the backend
    miwi_3d_xhr.open("DELETE", restQueryURL, true);
    miwi_3d_xhr.send();
  }

}

Utility functions

join_strings(strings_in, separator)

This function can be used to make a comma separated string from an array of strings.

function join_strings(strings_in, separator) { //: string
  /*
    strings_in string array
    separator string to be inserted between strings_in

    *result string - strings of strings_in separated by separator

    Example: join_strings(["ab", "cd", "ef"], ",") -> "ab,cd,ef"
  */
  var result, i;

  result = strings_in[0] || "";
  for (i = 1; i < strings_in.length; i++) {
    result = result + separator + strings_in[i];
  }

  return result;
}
set_accept_languages(http_request, languages)

This function is used to define preferred languages for query responses. The language preferences are coded to the Accept-Languages header of the http request.

set_accept_languages(http_request, languages) {
  /*
    This function creates an Accept-Languages header to the HTTP request.
    This must be called between http_request.open() and 
    http_request.send() .

    http_request - an instance of XMLHttpRequest
    languages    - string array containing the codes of the languages 
                   accepted in the response in descending priority. 
                   The ISO 639-1 language codes are used. If any language 
                   texts are accepted in case of none of the listed 
                   languages are found, an asterisk is used as the last 
                   code.
                   Example: ["en","fi","de","es","*"]
  */
  var i, q;

  q = 9;
  for (i = 0; i < languages.length; i++) {
    if (i == 0) {
      http_request.setRequestHeader('Accept-Language', languages[0]);
    } else {
      if (languages[i] != "") {
        http_request.setRequestHeader('Accept-Language', languages[i] +
            ';q=0.' + q);
        if (q > 1) {
          q--;
        }
      }
    }
  }
}