Display Geographic Locations Extracted from Photos

Contents

Overview

This article introduces a sample customization using OpenStreetMap (External link) and Openlayers (External link). In this example, geographic data are extracted from images (photos) attached to records of the Kintone App. Maps are displayed on the Record Details page and Record List page, with pins created from the extracted data.

OpenStreetMap and OpenLayers

OpenStreetMap (OSM) is a collaborative project to create a free editable map of the world. Maps from OpenStreetMap are freely available and accessible. OpenLayers is a JavaScript library for developing dynamic map widgets with tools for displaying and editing geographic information.

Sample Image

For single images on the Record Details page

If only one image is saved in the record, a map is displayed with a pin on the geographic location of the image data.

Screenshot of a map displayed in the record details page, with data extracted from one image.

For multiple images on the Record Details page

If multiple images are saved in the record, the map automatically adjusts the zoom levels to show all the pins.

Screenshot of a map displayed in the record details page, with data extracted from two images.

For images with no location data

If location info cannot be obtained, the following message is displayed.

Screenshot of the record details page, where the map could not be loaded due to lack of data.

For the Record List page

On the Record List page, the map is displayed in the header space and multiple pins are shown.

Screenshot of a map with multiple locations being displayed on the Record List page.

Prepare the App

Create the Form

Create an App (External link) that includes the following fields.

Field Type Field Name Field Code / Element ID
Attachment Picture pic
Blank space map

After creating the App, add a few records inside. Make sure to upload photos to the records.

Set the Libraries

CSS

Set the following URL into the App’s JavaScript and CSS Customization settings (External link), under the Upload CSS File for PC option.

  • https://js.kintone.com/openlayers/v3.17.1/ol.css
JavaScript

Set the following URLs into the App’s JavaScript and CSS Customization settings (External link), under the Upload JavaScript for PC option.

  • https://js.kintone.com/jquery/2.2.4/jquery.min.js
  • https://cdnjs.cloudflare.com/ajax/libs/blueimp-load-image/2.1.0/load-image.all.min.js
  • https://js.kintone.com/openlayers/v3.17.1/ol.js

Sample Code

Type the following code into a text editor and save it as a JavaScript file. Upload it to the App’s JavaScript and CSS Customization settings (External link).

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
(function() {
  'use strict';

  // Download the attached file to Kintone
  function getFile(url) {
    var df = new $.Deferred();
    var xhr = new XMLHttpRequest();

    xhr.open('GET', url, true);
    xhr.setRequestHeader('X-Requested-With', 'XMLHttpRequest');
    xhr.responseType = 'blob';

    xhr.onload = function(e) {
      if (this.status === 200) {
        df.resolve(this.response);
      }
    };

    xhr.send();
    return df.promise();
  }

  // Convert from minutes/seconds to percentage
  function toPercentage(ref, geo) {
    if (ref === 'N' || ref === 'E') {
      return geo[0] + geo[1] / 60 + geo[2] / 3600;
    } else if (ref === 'S' || ref === 'W') {
      return -(geo[0] + geo[1] / 60 + geo[2] / 3600);
    }
  }

  // Acquire EXIF coordinate information
  function getExif(imageData) {
    var df = new $.Deferred();
    // eslint-disable-next-line no-undef
    loadImage.parseMetaData(imageData, function(data) {
      // no EXIF data
      if (data.exif === undefined) {
        return df.resolve();
      }

      var gpsLatitude = data.exif.get('GPSLatitude');
      var gpsLatitudeRef = data.exif.get('GPSLatitudeRef');
      var gpsLongitude = data.exif.get('GPSLongitude');
      var gpsLongitudeRef = data.exif.get('GPSLongitudeRef');

      var latitude = toPercentage(gpsLatitudeRef, gpsLatitude);
      var longitude = toPercentage(gpsLongitudeRef, gpsLongitude);

      var position = {'longitude': longitude, 'latitude': latitude};

      df.resolve(position);
    });
    return df.promise();
  }

  // Convert latitude and longitude to spherical Mercator projection
  function convertCoordinate(longitude, latitude) {
    return ol.proj.transform([longitude, latitude], 'EPSG:4326', 'EPSG:3857');
  }

  // Create a layer to display markers
  function makeMarkerOverlay(coordinate) {
    var imgElement = document.createElement('img');
    var imgSrc = 'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABUAAAAZCAYAAADe1WXtAAAABHNCSVQICAgIfAhkiAAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAAAHwSURBVEiJpdW9a9NRFMbxT9qmNWmSRqlFhE6CLuIL1DcUHDqJm/0HBEUk3RQXpW7dXdysWlAEHQSlo4IoIohQKFJQxKWIIlTb5pc2anMdkmBbmleHZ3vul3Pvc865QggaCfu6uIOtTfkbGboYTRIN8yvJd5xoG4psmqk9LH0gBMJTQh+FJOPobAmKo718zbFcrACr+kI4TpThHQYbQhHr4WqG6MkG2FqtEsb5nWQJZ2pCMZDh5RBLc3WAa/WGsIMozV0k1kExnGR+jOKfJoFVLRBGKKT4jL0hBLq51kfhRZ2D04Qblfes5blNKcFynItwYTf5Ug3zc0KyolSlslrgIQpxxiCWYWaC0mbGUUIHASFDeFwD+Kjcbh/RWX3Tw1mixTYrzRP6iXBsXfppHl6m2M6bXqLYx4PNWmpnguhTi+m/L99iEf2bNn8P108RtQI9Qj5Ort5Ebenl27MmgfcoZZhFR93Zx8gu8o2GYIGQLYdzqKmFkuHtTVbrQXOsZJhsZUvtT1P4UQM4/S+cbS3t0xSTOVY2AkuEA+Q7Od/Okt6eID+7ATpRDmcGsba+k26unCRfBc6XRzXCwba+k0q18RRzUxXoOZZT3Kp3piG0Aj49SP41IcFPZP8bWmmxVwOsxDjbjL8pKPZ3c79eOGv1F5xHWAKxXNwiAAAAAElFTkSuQmCC';
    imgElement.setAttribute('src', imgSrc);

    var markerOverlay = new ol.Overlay({
      element: imgElement,
      position: coordinate,
      positioning: 'center-center'
    });

    return markerOverlay;
  }

  // Show the map and pin up
  function setPin(space, fileKeyList) {
    var map = new ol.Map({
      target: 'map',
      layers: [
        new ol.layer.Tile({
          source: new ol.source.OSM()
        })
      ],
      view: new ol.View({
        zoom: 15
      })
    });

    Promise.all(fileKeyList.map(function(fileKey) {
      var fileUrl = '/k/v1/file.json?fileKey=' + fileKey;
      return getFile(fileUrl);
    })).then(function(imageBlobList) {
      return Promise.all(imageBlobList.map(function(imageBlob) {
        return getExif(imageBlob);
      }));
    }).then(function(positionList) {
      var existPosition = false;

      var minLongitude = 180,
        minLatitude = 90;
      var maxLongitude = -180,
        maxLatitude = -90;
      for (var i = 0; i < positionList.length; i++) {
        var position = positionList[i];
        // no EXIF data
        if (position === undefined || position.longitude === undefined || position.latitude === undefined) {
          continue;
        }
        existPosition = true;

        var longitude = position.longitude;
        var latitude = position.latitude;
        var coordinate = convertCoordinate(longitude, latitude);
        var marker = makeMarkerOverlay(coordinate);
        map.addOverlay(marker);

        if (longitude < minLongitude) {
          minLongitude = longitude;
        }
        if (latitude < minLatitude) {
          minLatitude = latitude;
        }
        if (longitude > maxLongitude) {
          maxLongitude = longitude;
        }
        if (latitude > maxLatitude) {
          maxLatitude = latitude;
        }
      }

      if (existPosition === false) {
        $(space).text('Cannot display the map because location information could not be obtained');
        $(space).css('text-align', 'center').css('padding', '20px');
      } else if ((minLongitude === maxLongitude) && (minLatitude === maxLatitude)) {
        map.getView().setCenter(convertCoordinate(minLongitude, minLatitude));
      } else {
        // If there are multiple coordinates, calculate the center
        var extent = ol.proj.transformExtent([minLongitude, minLatitude, maxLongitude, maxLatitude],
          'EPSG:4326', 'EPSG:3857');
        map.getView().fit(extent, map.getSize());
      }
    }).catch(function(error) {
      console.log('ERROR', error);
    });
  }

  kintone.events.on('app.record.detail.show', function(event) {
    var record = event.record;

    var space = kintone.app.record.getSpaceElement('map');
    $(space).append('<div id="map" style="width:400px; height:400px"></div>');

    var fileKeyList = [];
    for (var i = 0; i < record.pic.value.length; i++) {
      var fileKey = record.pic.value[i].fileKey;
      fileKeyList.push(fileKey);
    }

    setPin(space, fileKeyList);
  });

  kintone.events.on('app.record.index.show', function(event) {
    // Delete once the map is already displayed
    if ($('div#map').length > 0) {
      $('div#map').remove();
    }

    var space = kintone.app.getHeaderSpaceElement();
    $(space).append('<div id="map" style="width:90%; height:400px"></div>');
    $('div#map').css('margin', '5px auto');

    var fileKeyList = [];
    for (var i = 0; i < event.records.length; i++) {
      var record = event.records[i];
      for (var j = 0; j < record.pic.value.length; j++) {
        var fileKey = record.pic.value[j].fileKey;
        fileKeyList.push(fileKey);
      }
    }

    setPin(space, fileKeyList);
  });

})();
caution
Attention

Caution: The order in which JavaScript and CSS are uploaded to an app matters. In this example, ensure that the jQuery, OpenStreetMap, and OpenLayers JavaScript imports are uploaded before the custom JavaScript file. You can change the order of uploads by clicking and dragging on the arrows for each item on the Upload JavaScript / CSS page.

After saving the settings and clicking on Update App, navigate to the Record Details page and the Record List page. Maps should be displayed with geographic locations retrieved from the images.

Screenshot of a world map displayed in the record details page, with data extracted from two images.

Notes

  • Photos attached from smartphones will not include geographic location data if the geolocation settings have been turned off.
  • Photos taken with the Take Photo or Video option on the Kintone Mobile App and Kintone accessed from mobile browsers do not include geographical location data.
  • Photos with reduced image file size may not include geographical location data.

Reference