GBDX

Vector Export Service Example Constructions

Scenarios of constructing Vector Export Service requests

All examples use Export Vectors API. (See Vector Export Service Reference Overview for specific details on the API.)

Note: Fields in the vector item 'attributes' and 'ingest_attributes' are moved to the top-level of a feature's 'properties' field. There is a chance for field collisions because of this. If multiple fields contain the same field name, the 'ingest_attributes' field supersedes both the 'attributes' field and the 'properties' field, and the 'attributes' field supersedes the 'properties' field. For example, in a vector containing an id field and an attributes.id field, the attribute.id field will overwrite the top-level id field on export for the given vector.

Constructing a request to export GeoJSON in zip files

The very minimum a zip export request needs are these parameters:

  • index name
  • request type

So the simplest request would be to export everything from a single index:

{
  "request_type": [ "zip" ],
  "index": "vector-my-index"
}

Note, if the index you're trying to export is large, your request could end up taking a very long time and result in a very large export file.

To help limit the number of documents returned and/or to tailor your export to the data you're particularly interested in, you can add a "query" entry to your request. The "query" entry is a JSON object that represents a valid Elasticsearch query (see the Elasticsearch query documentation for more details). For example, to limit the export to a certain item type, you could add a query like this:

{
  "request_type": [ "zip" ],
  "index": "vector-my-index",
  "query": {
    "query_string": {
      "query": "item_type: SENTINEL2"
    }
  }
}

To further refine your export, you may also want to only export items that intersect a specfic area of interest in the world. You can add a "geometry" field to specify that area using GeoJSON. For example, to limit the export to a simple polygon region, you could add a polygon geometry to the query request:

  "request_type": [ "zip" ],
  "index": "vector-my-index",
  "geometry": {
    "type": "Polygon",
    "coordinates": [[[0,0],[0,1],[1,1],[1,0],[0,0]]]
  }
}

You can also combine a query and a geometry to get very specific about what you want to export:

  "request_type": [ "zip" ],
  "index": "vector-my-index",
  "query": {
    "query_string": {
      "query": "item_type: SENTINEL2"
    }
  },
  "geometry": {
    "type": "Polygon",
    "coordinates": [[[0,0],[0,1],[1,1],[1,0],[0,0]]]
  }
}

VectorServices will export the data to a publicly accessible S3 bucket using a unique identifier as the directory prefix under that bucket. If you want to write data to a different location, you can specify that location using the following options:

  • s3_bucket: the bucket where the export artifacts should be written. If nothing is specified, then the files will be written to a default publicly-accessible S3 bucket.
  • s3_prefix: the prefix/directory name to use when writing the export data. If nothing is specified, VectorServices will assign a unique ID as the prefix.
  • s3_region: if the bucket you want to write to is not in us-east-1, you'll need to specify that region here.

A request using those options might look like this:

  "request_type": [ "zip" ],
  "index": "vector-my-index",
  "s3_bucket": "my_own_bucket",
  "s3_prefix": "my_meaningful_export",
  "s3_region": "us-west-1"
}

Finally, if you want to be notified immediately when the export finishes (instead of checking the status via VectorServices), you can provide an array of callback endpoints the export processor will send a message to when the export completes. If you configure a callback, at the minimum, you must provide a "url" field to indicate the endpoint to send the callback to. In addition, you can provide a "headers" object and a "params" object to indicate any custom headers or query string parameters the callback request might need. An export request with a configured callback might look like this:

  "request_type": [ "zip" ],
  "index": "vector-my-index",
  "callbacks": [
    {
      "url": "http://my.url.app/receiver",
      "headers": {
        "Authorization": "my credentials"
      },
      "params": {
        "my_param": "my_value"
      }
    }
  ]
}

Constructing a request to export Mapbox Vector Tiles

The very minimum a tile export request needs are these parameters:

  • index name
  • request type

The same options for queries, geometries, export locations, and callbacks all apply for tile requests, too. In addition, you need to supply some settings for the tile generator in a "tile_params" object.

At a minimum, in the "tile_params" object, you need to include the following fields:

  • tileset_name: A meaningful name for the tileset
  • layer_name: A meaningful name for the layer vectors will be written to in the tileset

So the simplest request would be to export tiles for everything from a single index might look like this:

  "request_type": [ "tile" ],
  "index": "vector-my-index",
  "tile_params": {
    "tileset_name": "My Tiles",
    "layer_name": "Some interesting features"
  }
}

Note, if the index you're trying to export is large, your request could end up taking a very long time.

In addition to those required tile parameters, you can provide a description for the tileset and tweak some parameters for the tile generation process:

  • tileset_description: A meaningful description for the tileset
  • min_zoom: the minimum zoom level to create tiles for (default: 0)
  • max_zoom: the maximum zoom level to create tiles for (default: guess based on input vectors)
  • simplification: the simplification factor for all zoom levels except the max zoom (default: 10)
  • drop_behavior: the behavior the tiler process uses to manage vectors if tiles become too big, for example, dropping the smallest or coalescing the densest. Default is to do nothing. Valid values include:
    • drop-densest-as-needed
    • drop-fraction-as-needed
    • drop-smallest-as-needed
    • coalesce-smallest-as-needed
    • coalesce-densest-as-needed
    • coalesce-fraction-as-needed
    • cluster-densest-as-needed

A request with those parameters set might look like this:

  "request_type": [ "tile" ],
  "index": "vector-my-index",
  "tile_params": {
    "tileset_name": "My Tiles",
    "layer_name": "Some interesting features",
    "tileset_description": "Features I think are interesting",
    "min_zoom": 5,
    "max_zoom": 10,
    "simplification": 8,
    "drop_behavior": "drop-densest-as-needed"
  }
}

Again, all the same query, geometry, and output location fields apply to tile export requests, too.

Constructing a request to export Shapefiles

The very minimum a shapefile export request needs are these parameters:

  • index name
  • request type

The same options for queries, geometries, export locations, and callbacks all apply for shapefile requests, too.

So the simplest request would be to export shapefiles for everything from a single index might look like this:

{
  "request_type": [ "shapefile" ],
  "index": "vector-my-index"
}

Note, if the index you're trying to export is large, your request could end up taking a very long time.

In addition to those required tile parameters, you can tweak some parameters for the shapefile generation process:

  • limited_field_count: Set to true to truncate the number of fields down to 250 for compatibility. (default: false)

A request with those parameters set might look like this:

  "request_type": [ "shapefile" ],
  "index": "vector-my-index",
  "shapefile_params": {
    "limited_field_count": true
  }
}

Again, all the same query, geometry, and output location fields apply to shapefile export requests, too.

Request multiple export types at once

It's possible to request multiple export types, such as both a tile export and a zip export, at the same time in the same request. Since the "request_type" field is an array, you simply need to specify all desired values in it. If the "tile" value is included, the request would include tile generation, and so you would also need to specify the required fields for the tile parameters as well. A minimal request would look like this:

  "request_type": [ "shapefile", "zip", "tile" ],
  "index": "vector-my-index",
  "tile_params": {
    "tileset_name": "My Tiles",
    "layer_name": "Some interesting features"
  }
}

Putting it all together

Finally, a request that includes every possible parameter would look like this:

  "request_type": [ "zip", "tile", "shapefile" ],
  "index": "vector-my-index",
  "query": {
    "query_string": {
      "query": "item_type: SENTINEL2"
    }
  },
  "geometry": {
    "type": "Polygon",
    "coordinates": [[[0,0],[0,1],[1,1],[1,0],[0,0]]]
  },
  "s3_bucket": "my_own_bucket",
  "s3_prefix": "my_meaningful_export",
  "s3_region": "us-west-1",
  "callbacks": [
    {
      "url": "http://my.url.app/receiver",
      "headers": {
        "Authorization": "my credentials"
      },
      "params": {
        "my_param": "my_value"
      }
    }
  ],
  "shapefile_params": {
    "limited_field_count": true
  },
  "tile_params": {
    "tileset_name": "My Tiles",
    "layer_name": "Some interesting features",
    "tileset_description": "Features I think are interesting",
    "min_zoom": 5,
    "max_zoom": 10,
    "simplification": 8,
    "drop_behavior": "drop-densest-as-needed"
  }
}

Vector Export Service Example Constructions

Scenarios of constructing Vector Export Service requests