GBDX

Lesson: Vector Export Service

Making Tilegen Requests

Before anything else, you must decide the kind of export you need. Tilegen offers both zipfile export of JSON files and Mapbox tiles. These can be exported with the same request. For reference on how to construct these requests, see Vector Export Service Example Constructions.

Once the request JSON is created, you can make a POST request to /insight-vector/api/export, using the JSON content as the body:

POST /insight-vector/api/export
Header: Content-Type: application/json

  "request_type": [ "zip" ],
  "index": "vector-my-index"
}

The service will respond with a request ID and a relative URL you can use to check the status of the request:

  "request_id": "0123456",
  "url": "/api/export/0123456/status"
}

Checking the Status of a Request

You can check the status of an export request by using the URL provided by the request submission response. In general, that URL will look like '/api/export/{request_id}/status'. For a "request_id" of "0123456", the call would look like this:

GET /insight-vector/api/export/0123456/status

The responses will differ in detail depending on how far along in the export process the task is. For a task that has just been submitted, the response would look like this:

  "request_id": "0123456",
  "parameters": { . . . the original request parameters . . . },
  "state": "submitted",
  "submitted_time": "yyyy-mm-ddThh:mm:ss",
  "success": false
}

Note, the "success" field will read "false" since the export has not yet successfully returned data.

Once the task has been picked up by the export processor and has started running, the response "state" field will change to "running", and a "start_time" field will be added

  "request_id": "0123456",
  "parameters": { . . . the original request parameters . . . },
  "state": "started",
  "submitted_time": "yyyy-mm-ddThh:mm:ss",
  "success": false,
  "start_time": "yyyy-mm-ddThh:mm:ss"
}

Finally, once the task has finished (successfully or not), the "state" will change to "finished", an "end_time" field will be added, along with a number of other fields that can be used for statistics and error reporting:

  "request_id": "0123456",
  "parameters": { . . . the original request parameters . . . },
  "state": "finished",
  "submitted_time": "yyyy-mm-ddThh:mm:ss",
  "success": true,
  "start_time": "yyyy-mm-ddThh:mm:ss"
  "end_time": "yyyy-mm-ddThh:mm:ss",
  "duration_seconds": {float},
  "duration_str": "hh:mm:ss",
  "query_duration_seconds": {float},
  "query_duration_str": "hh:mm:ss",
  "vectors_retrieved": {integer},
  "retrieval_errors": [
    "{any errors that occurred while retrieving data}"
  ],
  "tiler": {
      "output": "{the stdout/stderr of the tiler process . . . can be quite long}",
      "state": "{complete|failed}"
  },
  "tiler_duration_seconds": {float},
  "tiler_duration_str": "hh:mm:ss",
  "upload_duration_seconds": {float},
  "upload_duration_str": "hh:mm:ss",
  "upload_errors": [
    "{any errors that occurred during S3 tile upload}"
  ],
  "tileset_url": "{url}",
  "zip_duration_seconds": {float},
  "zip_duration_str": "hh:mm:ss",
  "zip_errors": [
    "{any errors that occurred during S3 zip upload}"
  ],
  "zip_files": [
    "{file name list}"
  ],
  "shapefile_duration_seconds": {float},
  "shapefile_duration_str": "hh:mm:ss",
  "shapefile_errors": [
    "{any errors that occurred during S3 shapefile upload}"
  ],
  "shapefile_files": [
    "{file name list}"
  ]
}

Using Generated Tiles

A successful tile generation export will provide a "tileset_url" field in the status response for the export task. You can use that URL as the source URL for a vector tile layer in a number of different Javascript libraries. For example, in mapbox-gl, you might be able to embed a URL like this:

"source": { "type": "vector", "tiles": ["http://cloudfront.base.url/my_output_prefix/{z}/{x}/{y}.pbf"] }

Note: QGIS supports streaming tiles. If you want to take your tiles offline for whatever reason, you can also download the files, keeping the file structure from S3, and view the vector tiles in QGIS if you have the correct plugin.

Downloading Zip Files

A successful zip file export will provide a list of zip files that were generated. You can retrieve those files from the S3 bucket and prefix listed in the status response.

Lesson: Vector Export Service