{"_id":"5b3fb1fb67cbc90003d2849a","project":"55faeacad0e22017005b8265","version":{"_id":"55faeacad0e22017005b8268","project":"55faeacad0e22017005b8265","__v":37,"createdAt":"2015-09-17T16:31:06.800Z","releaseDate":"2015-09-17T16:31:06.800Z","categories":["55faeacbd0e22017005b8269","55faf550764f50210095078e","55faf5b5626c341700fd9e96","55faf8a7825d5f19001fa386","560052f91503430d007cc88f","560054f73aa0520d00da0b1a","56005aaf6932a00d00ba7c62","56005c273aa0520d00da0b3f","5601ae7681a9670d006d164d","5601ae926811d00d00ceb487","5601aeb064866b1900f4768d","5601aee850ee460d0002224c","5601afa02499c119000faf19","5601afd381a9670d006d1652","561d4c78281aec0d00eb27b6","561d588d8ca8b90d00210219","563a5f934cc3621900ac278c","5665c5763889610d0008a29e","566710a36819320d000c2e93","56ddf6df8a5ae10e008e3926","56e1c96b2506700e00de6e83","56e1ccc4e416450e00b9e48c","56e1ccdfe63f910e00e59870","56e1cd10bc46be0e002af26a","56e1cd21e416450e00b9e48e","56e3139a51857d0e008e77be","573b4f62ef164e2900a2b881","57c9d1335fd8ca0e006308ed","57e2bd9d1e7b7220000d7fa5","57f2b992ac30911900c7c2b6","58adb5c275df0f1b001ed59b","58c81b5c6dc7140f003c3c46","595412446ed4d9001b3e7b37","59e76ce41938310028037295","5a009de510890d001c2aabfe","5a96f89c89442e002041144b","5b3f9b7267cbc90003d283a5"],"is_deprecated":false,"is_hidden":false,"is_beta":false,"is_stable":true,"codename":"v1","version_clean":"1.0.0","version":"1"},"category":{"_id":"5b3f9b7267cbc90003d283a5","project":"55faeacad0e22017005b8265","version":"55faeacad0e22017005b8268","__v":0,"sync":{"url":"","isSync":false},"reference":false,"createdAt":"2018-07-06T16:40:18.797Z","from_sync":false,"order":20,"slug":"vector-tile-generation-and-export","title":"Vector Export Service"},"user":"56267741db1eda0d001c3dbb","__v":0,"parentDoc":null,"updates":[],"next":{"pages":[],"description":""},"createdAt":"2018-07-06T18:16:27.157Z","link_external":false,"link_url":"","githubsync":"","sync_unique":"","hidden":false,"api":{"settings":"","results":{"codes":[]},"auth":"required","params":[],"url":""},"isReference":false,"order":2,"body":"| Table of Contents |\n| --- |\n| [Making Tilegen Requests](#section-making-tilegen-requests) |\n| [Checking the Status of a Request](#section-checking-the-status-of-a-request) |\n| [Using Generated Tiles](#section-using-generated-tiles) |\n| [Downloading Zip Files](#section-downloading-zip-files) |\n\n#Making Tilegen Requests#\nBefore anything else, you must decide the kind of export you need. Tilegen offers both zipfile export of JSON files and Mapbox tiles. These can be exported with the same request. For reference on how to construct these requests, see [Vector Export Service Example Constructions](doc:vector-export-service-example-constructions).\n\nOnce the request JSON is created, you can make a POST request to /insight-vector/api/export, using the JSON content as the body:\n\nPOST /insight-vector/api/export\nHeader: Content-Type: application/json\n\n```{\n  \"request_type\": [ \"zip\" ],\n  \"index\": \"vector-my-index\"\n}```\n\nThe service will respond with a request ID and a relative URL you can use to check the status of the request:\n\n```{\n  \"request_id\": \"0123456\",\n  \"url\": \"/api/export/0123456/status\"\n}```\n\n\n#Checking the Status of a Request#\n\nYou can check the status of an export request by using the URL provided by the request submission response.  In general, that URL will look like '/api/export/{request_id}/status'.  For a \"request_id\" of \"0123456\", the call would look like this:\n\nGET /insight-vector/api/export/0123456/status\n\nThe responses will differ in detail depending on how far along in the export process the task is.  For a task that has just been submitted, the response would look like this:\n\n```{\n  \"request_id\": \"0123456\",\n  \"parameters\": { . . . the original request parameters . . . },\n  \"state\": \"submitted\",\n  \"submitted_time\": \"yyyy-mm-ddThh:mm:ss\",\n  \"success\": false\n}```\n\nNote, the \"success\" field will read \"false\" since the export has not yet successfully returned data.\n\nOnce the task has been picked up by the export processor and has started running, the response \"state\" field will change to \"running\", and a \"start_time\" field will be added\n\n```{\n  \"request_id\": \"0123456\",\n  \"parameters\": { . . . the original request parameters . . . },\n  \"state\": \"submitted\",\n  \"submitted_time\": \"yyyy-mm-ddThh:mm:ss\",\n  \"success\": false,\n  \"start_time\": \"yyyy-mm-ddThh:mm:ss\"\n}```\n\nFinally, once the task has finished (successfully or not), the \"state\" will change to \"finished\", an \"end_time\" field will be added, along with a number of other fields that can be used for statistics and error reporting:\n\n```{\n  \"request_id\": \"0123456\",\n  \"parameters\": { . . . the original request parameters . . . },\n  \"state\": \"submitted\",\n  \"submitted_time\": \"yyyy-mm-ddThh:mm:ss\",\n  \"success\": true,\n  \"start_time\": \"yyyy-mm-ddThh:mm:ss\"\n  \"end_time\": \"yyyy-mm-ddThh:mm:ss\",\n  \"duration_seconds\": 500.0,\n  \"duration_str\": \"hh:mm:ss\",\n  \"query_duration_seconds\": 200.0,\n  \"query_duration_str\": \"hh:mm:ss\",\n  \"vectors_retrieved\": 1000000,\n  \"retrieval_errors\": [\n    \"{any errors that occurred while retrieving data}\"\n  ],\n  \"tiler\": {\n      \"output\": \"{the stdout/stderr of the tiler process . . . can be quite long}\",\n      \"state\": \"{complete|failed}\"\n  },\n  \"tiler_duration_seconds\": 200.0,\n  \"tiler_duration_str\": \"hh:mm:ss\",\n  \"zip_duration_seconds\": 30.0,\n  \"zip_duration_str\": \"hh:mm:ss\",\n  \"zip_errors\": [\n    \"{any errors that occurred during the zip file process}\"\n  ],\n  \"upload_duration_seconds\": 30.0,\n  \"upload_duration_str\": \"hh:mm:ss\",\n  \"upload_errors\": [\n    \"{any errors that occurred during S3 tile upload}\"\n  ],\n  \"tileset_url\": \"{url}\",\n  \"zip_files\": [\n    \"000000.zip\",\n    \"000001.zip\"\n  ]\n}```\n\n\n#Using Generated Tiles#\n\nA successful tile generation export will provide a \"tileset_url\" field in the status response for the export task.  You can use that URL as the source URL for a vector tile layer in a number of different Javascript libraries.  For example, in mapbox-gl, you might be able to embed a URL like this:\n\n```  \"source\": {\n      \"type\": \"vector\",\n      \"tiles\": [\"http://cloudfront.base.url/my_output_prefix/{z}/{x}/{y}.pbf\"]\n  }\n```\n\n**Note**: QGIS supports streaming tiles. If you want to take your tiles offline for whatever reason, you can also download the files, keeping the file structure from S3, and view the vector tiles in QGIS if you have the correct plugin.\n\n#Downloading Zip Files#\n\nA successful zip file export will provide a list of zip files that were generated.  You can retrieve those files from the S3 bucket and prefix listed in the status response.","excerpt":"","slug":"lesson-vector-export-service","type":"basic","title":"Lesson: Vector Export Service"}

Lesson: Vector Export Service


| Table of Contents | | --- | | [Making Tilegen Requests](#section-making-tilegen-requests) | | [Checking the Status of a Request](#section-checking-the-status-of-a-request) | | [Using Generated Tiles](#section-using-generated-tiles) | | [Downloading Zip Files](#section-downloading-zip-files) | #Making Tilegen Requests# Before anything else, you must decide the kind of export you need. Tilegen offers both zipfile export of JSON files and Mapbox tiles. These can be exported with the same request. For reference on how to construct these requests, see [Vector Export Service Example Constructions](doc:vector-export-service-example-constructions). Once the request JSON is created, you can make a POST request to /insight-vector/api/export, using the JSON content as the body: POST /insight-vector/api/export Header: Content-Type: application/json ```{ "request_type": [ "zip" ], "index": "vector-my-index" }``` The service will respond with a request ID and a relative URL you can use to check the status of the request: ```{ "request_id": "0123456", "url": "/api/export/0123456/status" }``` #Checking the Status of a Request# You can check the status of an export request by using the URL provided by the request submission response. In general, that URL will look like '/api/export/{request_id}/status'. For a "request_id" of "0123456", the call would look like this: GET /insight-vector/api/export/0123456/status The responses will differ in detail depending on how far along in the export process the task is. For a task that has just been submitted, the response would look like this: ```{ "request_id": "0123456", "parameters": { . . . the original request parameters . . . }, "state": "submitted", "submitted_time": "yyyy-mm-ddThh:mm:ss", "success": false }``` Note, the "success" field will read "false" since the export has not yet successfully returned data. Once the task has been picked up by the export processor and has started running, the response "state" field will change to "running", and a "start_time" field will be added ```{ "request_id": "0123456", "parameters": { . . . the original request parameters . . . }, "state": "submitted", "submitted_time": "yyyy-mm-ddThh:mm:ss", "success": false, "start_time": "yyyy-mm-ddThh:mm:ss" }``` Finally, once the task has finished (successfully or not), the "state" will change to "finished", an "end_time" field will be added, along with a number of other fields that can be used for statistics and error reporting: ```{ "request_id": "0123456", "parameters": { . . . the original request parameters . . . }, "state": "submitted", "submitted_time": "yyyy-mm-ddThh:mm:ss", "success": true, "start_time": "yyyy-mm-ddThh:mm:ss" "end_time": "yyyy-mm-ddThh:mm:ss", "duration_seconds": 500.0, "duration_str": "hh:mm:ss", "query_duration_seconds": 200.0, "query_duration_str": "hh:mm:ss", "vectors_retrieved": 1000000, "retrieval_errors": [ "{any errors that occurred while retrieving data}" ], "tiler": { "output": "{the stdout/stderr of the tiler process . . . can be quite long}", "state": "{complete|failed}" }, "tiler_duration_seconds": 200.0, "tiler_duration_str": "hh:mm:ss", "zip_duration_seconds": 30.0, "zip_duration_str": "hh:mm:ss", "zip_errors": [ "{any errors that occurred during the zip file process}" ], "upload_duration_seconds": 30.0, "upload_duration_str": "hh:mm:ss", "upload_errors": [ "{any errors that occurred during S3 tile upload}" ], "tileset_url": "{url}", "zip_files": [ "000000.zip", "000001.zip" ] }``` #Using Generated Tiles# A successful tile generation export will provide a "tileset_url" field in the status response for the export task. You can use that URL as the source URL for a vector tile layer in a number of different Javascript libraries. For example, in mapbox-gl, you might be able to embed a URL like this: ``` "source": { "type": "vector", "tiles": ["http://cloudfront.base.url/my_output_prefix/{z}/{x}/{y}.pbf"] } ``` **Note**: QGIS supports streaming tiles. If you want to take your tiles offline for whatever reason, you can also download the files, keeping the file structure from S3, and view the vector tiles in QGIS if you have the correct plugin. #Downloading Zip Files# A successful zip file export will provide a list of zip files that were generated. You can retrieve those files from the S3 bucket and prefix listed in the status response.