{"_id":"56e1cf39cd6a8d0e00d12176","project":"55faeacad0e22017005b8265","user":"56267741db1eda0d001c3dbb","__v":9,"version":{"_id":"55faeacad0e22017005b8268","project":"55faeacad0e22017005b8265","__v":32,"createdAt":"2015-09-17T16:31:06.800Z","releaseDate":"2015-09-17T16:31:06.800Z","categories":["55faeacbd0e22017005b8269","55faf550764f50210095078e","55faf5b5626c341700fd9e96","55faf8a7825d5f19001fa386","560052f91503430d007cc88f","560054f73aa0520d00da0b1a","56005aaf6932a00d00ba7c62","56005c273aa0520d00da0b3f","5601ae7681a9670d006d164d","5601ae926811d00d00ceb487","5601aeb064866b1900f4768d","5601aee850ee460d0002224c","5601afa02499c119000faf19","5601afd381a9670d006d1652","561d4c78281aec0d00eb27b6","561d588d8ca8b90d00210219","563a5f934cc3621900ac278c","5665c5763889610d0008a29e","566710a36819320d000c2e93","56ddf6df8a5ae10e008e3926","56e1c96b2506700e00de6e83","56e1ccc4e416450e00b9e48c","56e1ccdfe63f910e00e59870","56e1cd10bc46be0e002af26a","56e1cd21e416450e00b9e48e","56e3139a51857d0e008e77be","573b4f62ef164e2900a2b881","57c9d1335fd8ca0e006308ed","57e2bd9d1e7b7220000d7fa5","57f2b992ac30911900c7c2b6","58adb5c275df0f1b001ed59b","58c81b5c6dc7140f003c3c46"],"is_deprecated":false,"is_hidden":false,"is_beta":false,"is_stable":true,"codename":"v1","version_clean":"1.0.0","version":"1"},"category":{"_id":"56e1ccc4e416450e00b9e48c","project":"55faeacad0e22017005b8265","version":"55faeacad0e22017005b8268","pages":["56e1cdd892bf640e00b5564b","56e1cdfde416450e00b9e490","56e1ce4892bf640e00b5564e","56e1ce81e416450e00b9e494","56e1ceebe416450e00b9e497","56e1cf39cd6a8d0e00d12176","56e30b4b51857d0e008e77a3","56e30bd26e602e0e00700b16","56e30dd0d46bc30e007bb986","56e30e3d872bb20e0051ba39","56e30f91cb6ef20e0084f23c"],"__v":11,"sync":{"url":"","isSync":false},"reference":false,"createdAt":"2016-03-10T19:36:36.026Z","from_sync":false,"order":17,"slug":"aggregations-facets-vector-services-guide","title":"Aggregations & Facets Vector Services Guide"},"parentDoc":null,"updates":[],"next":{"pages":[],"description":""},"createdAt":"2016-03-10T19:47:05.466Z","link_external":false,"link_url":"","githubsync":"","sync_unique":"","hidden":false,"api":{"results":{"codes":[]},"settings":"","auth":"required","params":[],"url":""},"isReference":false,"order":3,"body":"<h2>Query Syntax</h2>\nVector querying uses Elasticsearch query syntax. For a full guide to Elasticsearch syntax, visit [Elasticsearch's syntax documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html#query-string-syntax).\n\nThings to note:\n  - Vector Services provides two representations of our vector data: GeoJSON and ESRI JSON. Our standard representation is GeoJSON, but we also provide ESRI-specific endpoints for users that require that format.\n \n  - When querying on items that are phrases, for instance \"Non-violent protest,\" the phrase must be in double quotation to ensure that it is the phrase that is being searched on and not the individual parts of the phrase. When querying items that require quotation, be sure to escape the quotes with a \"\\\". For example, querying for the item type \"Non-violent protest\" would be\n  <pre><code>item_type:\\\"Non-violent protest\\\"</code></pre>\n\n##Booleans##\n**Case matters in boolean search.** *Generally speaking, you want to capitalize your booleans in a boolean search.* Querying on `item_type:Road AND item_date:2017-01-13` searches ES for docs containing both the specified item_type and item_date, and will return only docs with the item_type of Road from the date of 2017-01-13. If, instead, the query is on `item_type:Road and item_date:2017-01-13`, the search in ES is for docs containing either the item_type of Road, the word and, or the item_date of 2017-01-13. In other words, `item_type:Road and item_date:2017-01-13` is translated as `item_type:Road OR and OR item_date:2017-01-13`.\n\n*Note*: Boolean matters inside parentheses as well - `item_type:(1BProduct and WV03_VNIR)` and `item_type:(1BProduct AND WV03_VNIR)` follows the same pattern, where the former breaks out into a search for `item_type:1BProduct OR item_type:and OR item_type:WV03_VNIR` while the latter breaks out into a search for `item_type:1BProduct AND item_type:WV03_VNIR`.\n  \n<h2>The Different \"Types\"</h2>\nIn the JSON of a given vector, there are multiple \"types.\" One type is grouped under the geometry section of the vector data.\n   <pre><code>{\n    \"geometry\": {\n      \"coordinates\": [\n        [\n          [\n            37.51445220776068,\n            55.89066246220917\n          ],\n          [\n            37.51445220776068,\n            55.89101403267135\n          ],\n          [\n            37.51510527269151,\n            55.89101403267135\n          ],\n          [\n            37.51510527269151,\n            55.89066246220917\n          ],\n          [\n            37.51445220776068,\n            55.89066246220917\n          ]\n        ]\n      ],\n      \"type\": \"Polygon\"\n    },</code></pre>\nAlthough a user cannot query specifically on that field, they can query on the synthetic field \"geom_type\" that corresponds to that field if the user is looking for results of a specific geometry.\n   <pre><code>geom_type:Polygon</code></pre>\n\nAnother type is the type of the vector item, listed as \"item_type\" under the properties section of the vector data.\n   <pre><code>\"properties\": {\n      \"ingest_source\": \"ObjectDetection\",\n      \"access\": {\n        \"groups\": [\n          \"_ALL_\"\n        ],\n        \"users\": [\n          \"_ALL_\"\n        ]\n      },\n      \"item_date\": \"2015-01-06T08:26:39Z\",\n      \"original_crs\": \"EPSG:4326\",\n      \"item_type\": [\n        \"Airliner\"\n      ],\n    </code></pre>\nThis type is queried as the \"item_type\" if the user is looking for results of a specific type of vector item. Generally speaking, a user querying on the type of vector item will get fewer - but more specific - results than a user querying on the type of geometry.\n   <pre><code>item_type:Airliner</code></pre>\n\nFinally, all of the data in vector services is represented as GeoJSON \"Feature\" objects.  An object's class is specified with a \"type\" attribute in the GeoJSON representation, so there will always be a top-level \"type\" field with a value of \"Feature\".\n   <pre><code>[\n  {\n    \"type\": \"Feature\",</code></pre> \n\n<h2>Property Fields</h2>\nThe following is a list of common property fields in the Vector Services. These fields exist for every properly ingested Vector Services vector item, and each of these fields may be used, separately or in combination, to construct more detailed and specific queries:\n - attributes\n - format\n - geom_type\n - ingest_attributes\n - ingest_date\n - ingest_source\n - item_date\n - item_type\n - name\n - source\n - text\n\nThe geom_type is automatically generated based on the item geometry. For a list of geom_types the Vector Services handles, see [Elasticsearch's GeoJSON Type](https://www.elastic.co/guide/en/elasticsearch/reference/2.0/geo-shape.html#input-structure) (note that Vector Services works with the GeoJSON Type in the table, not the Elasticsearch Type, and therefore does not handle 'envelope' or 'circle.').\n\n<h3>Source vs Ingest Source</h3>\nThe source is not a required field, whereas the ingest_source is. The source field is potentially the original source of the data, and can be used as a means to keep track of that as long as the one(s) who ingest the data include the information. For instance, for some HGIS data, the \"source\" could be Google Maps or Bing, but the \"ingest_source\" is HGIS. Simply put, \"ingest_source\" is where the data got into the vector services system, while \"source\" is where the data was actually created. In some cases, the values can be the same; in some cases, the values could be different.\n\n<h3>Ingest Date vs Item Date</h3>\nThe ingest_date represents the date that the vector has been written into Vector Services. The item_date represents the date that the vector item was originally generated. These date values can be identical in some cases, but in some cases they are not. When generating a query with a specific date, be sure to choose the correct date type.\n  - Date queries may be done using the ISO-8601 date format (YYYY-MM-DDT00:00:00.000Z ex: 2016-31-01T14:12:28.011Z) or the \"now\" format. The \"now\" format can be constructed using the following time ranges:\n  <pre><code>y (year)\n   M (month)\n   w (week)\n   d (day)\n   h (hour)\n   m (minute)\n   s (second)</code></pre>\n  -  The date expressions can include simple \"date math\" by appending \"+\" or \"-\" along with a numeral and a unit.\n  For example, if a user wants to query on the items ingested over the last week, the query in \"now\" format would be\n\n  <pre><code>ingest_date:[now-1w TO now]</code></pre>\nThe \"now\" format is also a moving range, whereas the ISO-8601 format is static.  **Note**: the capital 'M' is for months and lower-case 'm' is for minutes.  It's very easy to confuse these when entering date phrases, so if the results are not as expected (e.g. empty aggregations), check to make sure the time phrase is using the correct units.\n\n<h3>Attributes and Ingest Attributes Fields</h3>\nThe attributes and ingest_attributes fields, unlike the other fields listed, have subfields to query on. Different vector items have different attribute and/or ingest_attribute fields depending on the source data. To query on an attribute or ingest_attribute subfield, the user must use additional formatting, using \"attributes.\" or \"ingest_attributes.\" as a prefix. For instance, suppose there is an attribute field called \"userName\" with the string \"User1\" in a few documents that the user wants to query on. To see those documents, the user queries on\n  <pre><code>attributes.userName:User1</code></pre>\n  \n  - For a list of DG Catalog-specific attribute fields to query on, see [DG Catalog Attributes](doc:dgcatalog-attributes).\n  - For a list of ObjectDetection-specific attribute fields to query on, see [ObjectDetection Attributes](doc:objectdetection-attributes).\n  - For a list of Twitter-specific attribute fields to query on, see [Twitter Attributes](doc:twitter-attributes).\n\n<h2>Type Suffixes</h2>\n<h3>Analyzed vs Not Analyzed</h3>\nString fields may either be analyzed or not analyzed on ingest. Analyzed fields are broken apart into \"tokens\" based on punctuation and whitespace, and everything is made lowercase. So, if a field is analyzed, the user may query on it and retrieve any results that include any part of the queried field. For example, suppose there is an analyzed field with the string \"Google FTW\" in one document, and \"www.google.com\" in another document; the user queries for \"google\" on this field and will retrieve both documents.\n\nFields not analyzed will only return items that exactly match the query parameters. In the prior example, supposing that the field is not analyzed in this case, the query for \"google\" would return neither document.  However, a query on the exact term \"www.google.com\" would return the second document.\n\nThe following fields are ingested in analyzed form, and do not require the analyzed suffix.\n - name\n - text\n\nIn order to query on a field that is not analyzed by default, the user should add the suffix .analyzed to the field query. For example,\n  <pre><code>item_type:building</code></pre>\nIn the above case, since item_type is not analyzed, the returned results would only be items with the exact type \"building\" - case sensitive. In the following case, adding the .analyzed suffix returns results that include building in the type, case insensitive.\n\n  <pre><code>item_type.analyzed:building</code></pre>\nIn this case, results of both 'BUILDING' and 'building' would be returned, along with items with an item_type of, for example, \"Government Building,\" etc.\n\n<h3>Raw</h3>\nUsers may want to query a string attribute as a not-analyzed value. In that case, querying by adding the raw suffix to the attribute field name will query the non-tokenized content of that field, including whitespaces and anything following. Note, however, that only string attribute fields have a '_raw' version. For instance, adding the raw suffix to a date field will return an empty set, because there are no raw date fields.\n\nIn order to query on a raw field, the user should add the suffix _raw to the end of the field they are querying on. For example,\n  <pre><code>terms:attributes.userName</code></pre>\nIn the above case, since the userName field is not raw, the returned results would potentially return abridged results due to whitespace. (ie, userName \"John Smith\" would return as \"John\")\n\n  <pre><code>terms:attributes.userName_raw</code></pre>\nIn this case, the user would retrieve the full \"John Smith\" as one of the results for userName.","excerpt":"Syntax information for Aggregation and Vector Services calls","slug":"vector-query-syntax-query-fields-and-type-suffixes","type":"basic","title":"Vector Query Syntax, Query Fields, and Type Suffixes"}

Vector Query Syntax, Query Fields, and Type Suffixes

Syntax information for Aggregation and Vector Services calls

<h2>Query Syntax</h2> Vector querying uses Elasticsearch query syntax. For a full guide to Elasticsearch syntax, visit [Elasticsearch's syntax documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html#query-string-syntax). Things to note: - Vector Services provides two representations of our vector data: GeoJSON and ESRI JSON. Our standard representation is GeoJSON, but we also provide ESRI-specific endpoints for users that require that format. - When querying on items that are phrases, for instance "Non-violent protest," the phrase must be in double quotation to ensure that it is the phrase that is being searched on and not the individual parts of the phrase. When querying items that require quotation, be sure to escape the quotes with a "\". For example, querying for the item type "Non-violent protest" would be <pre><code>item_type:\"Non-violent protest\"</code></pre> ##Booleans## **Case matters in boolean search.** *Generally speaking, you want to capitalize your booleans in a boolean search.* Querying on `item_type:Road AND item_date:2017-01-13` searches ES for docs containing both the specified item_type and item_date, and will return only docs with the item_type of Road from the date of 2017-01-13. If, instead, the query is on `item_type:Road and item_date:2017-01-13`, the search in ES is for docs containing either the item_type of Road, the word and, or the item_date of 2017-01-13. In other words, `item_type:Road and item_date:2017-01-13` is translated as `item_type:Road OR and OR item_date:2017-01-13`. *Note*: Boolean matters inside parentheses as well - `item_type:(1BProduct and WV03_VNIR)` and `item_type:(1BProduct AND WV03_VNIR)` follows the same pattern, where the former breaks out into a search for `item_type:1BProduct OR item_type:and OR item_type:WV03_VNIR` while the latter breaks out into a search for `item_type:1BProduct AND item_type:WV03_VNIR`. <h2>The Different "Types"</h2> In the JSON of a given vector, there are multiple "types." One type is grouped under the geometry section of the vector data. <pre><code>{ "geometry": { "coordinates": [ [ [ 37.51445220776068, 55.89066246220917 ], [ 37.51445220776068, 55.89101403267135 ], [ 37.51510527269151, 55.89101403267135 ], [ 37.51510527269151, 55.89066246220917 ], [ 37.51445220776068, 55.89066246220917 ] ] ], "type": "Polygon" },</code></pre> Although a user cannot query specifically on that field, they can query on the synthetic field "geom_type" that corresponds to that field if the user is looking for results of a specific geometry. <pre><code>geom_type:Polygon</code></pre> Another type is the type of the vector item, listed as "item_type" under the properties section of the vector data. <pre><code>"properties": { "ingest_source": "ObjectDetection", "access": { "groups": [ "_ALL_" ], "users": [ "_ALL_" ] }, "item_date": "2015-01-06T08:26:39Z", "original_crs": "EPSG:4326", "item_type": [ "Airliner" ], </code></pre> This type is queried as the "item_type" if the user is looking for results of a specific type of vector item. Generally speaking, a user querying on the type of vector item will get fewer - but more specific - results than a user querying on the type of geometry. <pre><code>item_type:Airliner</code></pre> Finally, all of the data in vector services is represented as GeoJSON "Feature" objects. An object's class is specified with a "type" attribute in the GeoJSON representation, so there will always be a top-level "type" field with a value of "Feature". <pre><code>[ { "type": "Feature",</code></pre> <h2>Property Fields</h2> The following is a list of common property fields in the Vector Services. These fields exist for every properly ingested Vector Services vector item, and each of these fields may be used, separately or in combination, to construct more detailed and specific queries: - attributes - format - geom_type - ingest_attributes - ingest_date - ingest_source - item_date - item_type - name - source - text The geom_type is automatically generated based on the item geometry. For a list of geom_types the Vector Services handles, see [Elasticsearch's GeoJSON Type](https://www.elastic.co/guide/en/elasticsearch/reference/2.0/geo-shape.html#input-structure) (note that Vector Services works with the GeoJSON Type in the table, not the Elasticsearch Type, and therefore does not handle 'envelope' or 'circle.'). <h3>Source vs Ingest Source</h3> The source is not a required field, whereas the ingest_source is. The source field is potentially the original source of the data, and can be used as a means to keep track of that as long as the one(s) who ingest the data include the information. For instance, for some HGIS data, the "source" could be Google Maps or Bing, but the "ingest_source" is HGIS. Simply put, "ingest_source" is where the data got into the vector services system, while "source" is where the data was actually created. In some cases, the values can be the same; in some cases, the values could be different. <h3>Ingest Date vs Item Date</h3> The ingest_date represents the date that the vector has been written into Vector Services. The item_date represents the date that the vector item was originally generated. These date values can be identical in some cases, but in some cases they are not. When generating a query with a specific date, be sure to choose the correct date type. - Date queries may be done using the ISO-8601 date format (YYYY-MM-DDT00:00:00.000Z ex: 2016-31-01T14:12:28.011Z) or the "now" format. The "now" format can be constructed using the following time ranges: <pre><code>y (year) M (month) w (week) d (day) h (hour) m (minute) s (second)</code></pre> - The date expressions can include simple "date math" by appending "+" or "-" along with a numeral and a unit. For example, if a user wants to query on the items ingested over the last week, the query in "now" format would be <pre><code>ingest_date:[now-1w TO now]</code></pre> The "now" format is also a moving range, whereas the ISO-8601 format is static. **Note**: the capital 'M' is for months and lower-case 'm' is for minutes. It's very easy to confuse these when entering date phrases, so if the results are not as expected (e.g. empty aggregations), check to make sure the time phrase is using the correct units. <h3>Attributes and Ingest Attributes Fields</h3> The attributes and ingest_attributes fields, unlike the other fields listed, have subfields to query on. Different vector items have different attribute and/or ingest_attribute fields depending on the source data. To query on an attribute or ingest_attribute subfield, the user must use additional formatting, using "attributes." or "ingest_attributes." as a prefix. For instance, suppose there is an attribute field called "userName" with the string "User1" in a few documents that the user wants to query on. To see those documents, the user queries on <pre><code>attributes.userName:User1</code></pre> - For a list of DG Catalog-specific attribute fields to query on, see [DG Catalog Attributes](doc:dgcatalog-attributes). - For a list of ObjectDetection-specific attribute fields to query on, see [ObjectDetection Attributes](doc:objectdetection-attributes). - For a list of Twitter-specific attribute fields to query on, see [Twitter Attributes](doc:twitter-attributes). <h2>Type Suffixes</h2> <h3>Analyzed vs Not Analyzed</h3> String fields may either be analyzed or not analyzed on ingest. Analyzed fields are broken apart into "tokens" based on punctuation and whitespace, and everything is made lowercase. So, if a field is analyzed, the user may query on it and retrieve any results that include any part of the queried field. For example, suppose there is an analyzed field with the string "Google FTW" in one document, and "www.google.com" in another document; the user queries for "google" on this field and will retrieve both documents. Fields not analyzed will only return items that exactly match the query parameters. In the prior example, supposing that the field is not analyzed in this case, the query for "google" would return neither document. However, a query on the exact term "www.google.com" would return the second document. The following fields are ingested in analyzed form, and do not require the analyzed suffix. - name - text In order to query on a field that is not analyzed by default, the user should add the suffix .analyzed to the field query. For example, <pre><code>item_type:building</code></pre> In the above case, since item_type is not analyzed, the returned results would only be items with the exact type "building" - case sensitive. In the following case, adding the .analyzed suffix returns results that include building in the type, case insensitive. <pre><code>item_type.analyzed:building</code></pre> In this case, results of both 'BUILDING' and 'building' would be returned, along with items with an item_type of, for example, "Government Building," etc. <h3>Raw</h3> Users may want to query a string attribute as a not-analyzed value. In that case, querying by adding the raw suffix to the attribute field name will query the non-tokenized content of that field, including whitespaces and anything following. Note, however, that only string attribute fields have a '_raw' version. For instance, adding the raw suffix to a date field will return an empty set, because there are no raw date fields. In order to query on a raw field, the user should add the suffix _raw to the end of the field they are querying on. For example, <pre><code>terms:attributes.userName</code></pre> In the above case, since the userName field is not raw, the returned results would potentially return abridged results due to whitespace. (ie, userName "John Smith" would return as "John") <pre><code>terms:attributes.userName_raw</code></pre> In this case, the user would retrieve the full "John Smith" as one of the results for userName.