GBDX

Workflow API Course

Overview of the the GBDX Workflow system

A "workflow" is a series of tasks chained together to run on the GBDX platform. Each "task" is an individual process that performs a specific action. In order to run as part of a workflow, a task must be registered in the workflow system's task registry.

Tasks are run from Docker containers that are available via Docker Hub. User inputs must be validated against the system's task and workflow JSON schemas. When a workflow is run, the system generates events that indicate status.

For more information on how tasks integrate into the workflow system, read our
Task and Workflow Course

Workflow Resources

The Workflow API allows users to:

  • register a task in the workflow system's task registry
  • manage tasks in the task registry
  • create a new workflow by submitting a workflow definition
  • see the status of a workflow
  • search for workflows by status, time range, or owner.
Resource Description
Workflows A series of tasks chained together to complete an action
Tasks Atomic activities that perform a given action. They have one or more inputs and can have one or more outputs
Workflow Events System status events generated by the system, the events for a task include submission, starting, running and completing
Schemas JSON schemas used to validate user inputs. The two schemas are the task definition schema and the workflow schema

Registering a Task on GBDX

If you are an algorithm producer, you will need to register your task on GBDX. This requires you to "Dockerize" your task and submit a JSON task definition to the GBDX Workflow system.

To learn how to create and Dockerize a task to run on GBDX, see How to Create a Task

Add "tdgpdeploy" as a Collaborator

Before you register your task in the GBDX task registry (see below), you'll need to add "tdgpdeploy" as a collaborator in Docker Hub. Depending on the type of Docker Hub account you have, you'll either add it as a direct collaborator or as a member of your read-only access collaborator team.

This example shows how to add "tdgpdeploy" as a direct collaborator. Type the name in the "username" box and click "add user". It will appear in the user list with "Collaborator" access.

If your Docker Hub account level includes organizations, refer to the
Docker Hub documentation to learn how to add tdgpdeploy to your readonly access team.

Register the Task in the GBDX Task Registry

Tasks must be registered with the workflows task registry before they can be added to a workflow. If you are using a task that's already been registered on the GBDX platform, you can skip this step.

To register a task, the definition JSON is created and and sent as a Body POST to the /workflows/v1/tasks endpoint.

Rules for Tasks

  • Tasks require a name
  • Tasks require a version number. When registering a new version of a task, the version number
    must be incremented.
  • When a new task is registered, the "isPublic" task must be set to "false" or not included in the task definition. Only a superuser can set a task to "public".
    *If a new version of a public task is registered, the owner can set the new version to public.
  • Tasks require at least one input port and one output port.
  • Task ports require a name and a type
  • Tasks require one container. Only Docker is currently supported.

All tasks must be defined and registered in the GBDX Task Registry. The Task definition JSON file includes the following sections:

Create the Task Definition

The task definition has four main sections:

  1. Input Port Descriptors
  2. Output Port Descriptors
  3. Container Descriptors
  4. Name and Properties

Example Task Definition

{
    "inputPortDescriptors": [
        {
            "required": true,
            "description": "A string input.",
            "name": "inputstring",
            "type": "string"
        },
        {
            "name": "dependency_input",
            "type": "string"
        }
    ],
    "outputPortDescriptors":[
        {
            "name": "dependency_output",
            "type": "string"
        }
    ],
    "containerDescriptors": [
        {
            "type": "DOCKER",
            "command": "",
            "properties": {
                "image": "tdgp/test-success",
                "mounts": [
                    {
                        "local": "$task_data_dir",
                        "container": "/mnt/work",
                        "read_only": false
                    }
                ]
            }
        }
    ],
    "description": "Runs a no-op task that writes successful output status.",
    "name": "test-success",
    "version": "0.0.1",
    "taskOwnerEmail": "TEST@TEST.com",
    "properties": {
			"isPublic": false,
			"timeout": 7200,
    	"authorizationRequired": true
    }
	}

Name, Version, Description, and Properties

Element Description
Name The name you define for the task
Version The version number for the task. New tasks default to 0.0.1 if the version is not defined.
taskOwnerEmail This should be the email address of the technical owner of the task. This field is required when registering a new task or a new version of a task. See How to Version a Task
Description A brief description of the task
Properties See below

The properties section includes the following:

Property Description
isPublic Boolean value that determines whether a task is private or public.
timeout The task will time out if not completed within the set time frame. The timeout value is in seconds. See below for minimum, default, and maximum values allowed.
authorizationRequired Boolean value. If authorizationRequired is True, the task will capture the requestor's GBDX token and make GBDX requests on their behalf. If the task has impersonation_allowed is True in the Workflow definition, authorizationRequired is required to be True. See Task User Impersonation for details.

Note: The properties section is not required. If the properties are not included as part of the task definition the default values will be used. See below for default values. However, if the properties section is part of your task definition, the "timeout" property is required and a value must be set.

Example:

Private and Public tasks

These are the business rules for the "isPublic" flag:

  • New tasks must be registered as private tasks.
  • Only a member of the GBDX team at DigitalGlobe can change a new task from "private" to "public".
    If a new version of a public task is published, the task owner can set the new version to public. The task owner can change a task from "public" to "private".

When a new task is registered on GBDX, it must be set to "private". "Private" means only users from the account that registered the task can run it.

Definitions

Value Description
Public Any GBDX user can access or use this task, regardless of what account they're associated with.
Private Only users associated with the account the task is registered under can access or use this task. A "private" task cannot be shared between multiple accounts.

To register a private task:

In the JSON task definition, under "properties", set "isPublic": false

If the properties section is not included in the task definition, the task will default to "isPublic": false.

Flag Settings

Setting Result
"isPublic": true The task will be public
"isPublic": false the task will be private
"isPublic": [no value] The default value of "false" will be used. The task will be private.
'isPublic" not included in the task definition The default value of "false" will be used. The task will be private.

Converting a task to "public".

Before a task can be set to public, a review process is required. Setting a task to "public" is part of the algorithm submission process. The task is set to public by the GBDX team once it has been reviewed.

Error Conditions

Submitting a task definition for a new task with isPublic":true will result in the following error, and the task will not be registered.

Status code:403, Message: 'Creating a public task is unauthorized.'

Define a Time-out threshold for a Task

By default, tasks time out after 7200 seconds (2 hours).

The default time out value can be changed in the properties section of the task definition jSON. You can do this by setting a value in seconds for "timeout": [value].

Type Value
Default 7200 (2 hours)
Minimum 0
Maximum 172800 (48 hours)

Important: If the "properties" section is included in the task definition, the "timeout" property is required and a value must be set.

Updating the Task Definition

All fields in the task definition except Name and Version can be updated in a PUT request. A PUT request does not require the version to be incremented, and it does not trigger a task Docker migration.

Make a PUT request to the /workflows/v1/tasks endpoint. Include the full task definition, including the changes.

Input Port Descriptors

Use this section to define the task input ports for this task.

Input ports have:

Element Value
Name The input port name
Type input port type (string, directory)
Description A short description of the input type for this port
required boolean

Output Port Descriptors

Use this section to define the task output ports for this task.

Element Value
Name The output port name
Type output port type (string, directory)
Description A short description of the output type for this port

Important: Symbolic links should not be written as task output.

Container Descriptors

Use this section to define the container in which the task is run. Only Docker containers are supported on GBDX.

Element Value
Type The type of container. Currently only Docker is supported.
Domain If this property is specified, the task must always run on this domain. The default domain is is an AWS R4.2xlarge machine. For a complete list of available domains on GBDX, see GBDX Supported Domains
Image The value for this property will be the name of the Docker image that is pulled from Docker Hub
Command The command to run within the container to initiate the task.

To see the task descriptor schema, run Task Definition Schema

Search for a Workflow

You can search for workflows using the following criteria.

Filter Description
Search by State The state of the workflow. For example submitted, failed, timedout, succeeded. You can only search by one state at a time.
Search by Time Range The lookback time (last N hours) is based on the time the workflow search query is run. For example, if a query is ran at 14:00 and the value is set to 2, the workflows that started at 12:00 and on will be listed. The default value is 24.
Search by Owner Search by workflow owner will return all workflows that have the specific owner listed.

Run a Workflow

To run a workflow, make a POST request to https://geobigdata.io/workflows/v1/workflows with the workflow definition in the request body. The request will list each task that is to be run in the workflow, in the order it should be run, going from top to bottom. The request will include a workflow name, and the task name, task type, and inputs and outputs for each task.

To see the workflow definition schema, see Workflow Definition Schema .
To see an example workflow definition as a POST request body, see Submit a Workflow

Retrieve STDOUT output from a completed task in a workflow

GET to /workflows/v1/workflows/<workflow_id>/tasks/<task_id>/stdout

Response: 200 OK
Response body contains the STDOUT from the Docker container that ran the task.
When there is no data, the request will return "empty."

Retrieve STDERR output from a completed task in a workflow

GET to /workflows/v1/workflows/<workflow_id>/tasks/<task_id>/stderr

Response: 200 OK
Response body contains the STDERR from the Docker container that ran the task.
When there is no data, the request will return "empty."

Task User Impersonation

Tasks can perform GBDX operations on behalf of a user. For example, a task can make a catalog query, place an order, or use any other GBDX API. This means a task can run in a workflow without authorization.

In order for a task to performs GBDX operations on behalf of the user, the task definition and the workflow definition must allow it.

To allow user impersonation:

  • In the task definition, "authorizationRequired" must be set to true (false is the default if the value is not set).

  • In the workflow definition, "impersonation_allowed" must be set to true (false is the default if the value is not set).

When impersonation is allowed, the user token is passed as part of the runtime information, and will be available in gbdx_runtime.json in user_token field.

Task Definition

Name Description Type Default
authorizationRequired Indicator that the task logic will require GBDX authorization from the running user. If this is true and during workflow invocation 'impersonation_allowed' flag is not set on the task, workflow will fail. boolean false

While the task descriptor schema has changed, tasks that are already registered are not updated with the authorizationRequired flag. These tasks, and any new tasks registered without the flag, will default to "false."

To add the flag or change the value for an existing task, the task must be re-registered.

This example shows a task descriptor with "authorizationAllowed" = true.

{
 "inputPortDescriptors": [
   {
     "name": "dummy",
     "type": "string"
   }
 ],
 "containerDescriptors": [
   {
     "type": "DOCKER",
     "command": "",
     "properties": {
       "image": "dummy"
     }
   }
 ],
 "description": "Test task",
 "name": "test-task-with-required-authorization",
 "properties": {
   "timeout": 7200,
   "authorizationRequired": true
 }
}

Workflow Definition

Name Description Type Default
impersonation_allowed when set to true, the task can use user security token. Impersonation is allowed. boolean false

When there are multiple tasks defined in a single workflow, the tasks can have different values and the workflow will run successfully. This example shows a workflow definition with three tasks. Two allow impersonation and one does not.

{
    "name": "AOP_savetoS3",
    "tasks": [
        {
            "name": "AOP",
            "outputs": [
                {
                    "name": "data"
                },
                {
                    "name": "log"
                }
            ],
            "inputs": [
                {
                    "name": "data",
                    "value": "<bucket location>"
                },
                {
                    "name": "enable_acomp",
                    "value": "true"
                }
            ],
            "taskType": "AOP_Strip_Processor",
            "impersonation_allowed": false
        },
        {
            "name": "SavetoS3_Data",
            "inputs": [
                {
                    "name": "data",
                    "source": "AOP:data"
                },
                {
                    "name": "destination",
                    "value": "s3://<your bucket location>"
                },
                { 
                     "name":"session_token",
                     "value":"<your_aws_session_token>"
                },
                { 
                     "name":"access_key_id",
                     "value":"<your_aws_access_key_id>"
                },
                { 
                     "name":"secret_key",
                     "value":"<your_aws_secret_key>"
                }
            ],
            "taskType": "SaveToS3",
            "impersonation_allowed": true
        }
    ]
}

Scenarios

Number Scenario Result
Scenario #1 Task Definition "authorizationRequired" = true; Workflow Definition task impersonation_allowed = true Task runs successfully
Scenario #2 Task Definition "authorizationRequired" = false; Workflow Definition task impersonation_allowed = false Task runs successfully
Scenario #3 Task Definition "authorizationRequired" = true; Workflow Definition task impersonation_allowed = false Workflow is rejected with an error that task requires impersonation
Scenario #4 Task Definition "authorizationRequired" = false; Workflow Definition task impersonation_allowed = true Task runs successfully

Batch Workflows

This example is from a batch workflow named "batch_test." This is the batch value list for the input port named "input_data."

      {
       "name": "batch_test",
       "batch_values": [
           {
               "name": "input_data",
               "values": [
                   "CAT1",
                   "CAT2",
                   "CAT3"
               ]
           }

This is the old method for defining a batch value for a port descriptor. With this breaking change, this format will cause an error.

 "inputs": [
           {
             "name": "data",
             "value": "$batch_value:input_data"
            }

This is the new method for defining a batch value for a port descriptor:

  
     "inputs": [
           {
             "name": "data",
             "value": "{{input_data}}"
           }

Batch workflow overview

With batch workflows, you can submit multiple workflows with one API request. A batch workflow request spawns multiple workflows that run the same tasks on multiple inputs concurrently.

In a batch workflow request, a single workflow definition is created, but multiple input values are allowed for an input port. Each input value is processed as a separate workflow. Each instance will have its own workflow ID.

In a batch workflow definition, output ports can also have batch values. When this is true, the batch value list for the output port descriptor must have the same number of values as the input ports for that task.

Batch Workflow API

The API endpoint for submitting batch workflows is

/workflows/v1/batch_workflows

This endpoint does not replace workflows/v1/workflows. Use this endpoint to submit a single workflow request.

The batch workflow endpoint, workflows/v1/batch_workflows, accepts multiple values for task inputs in one submission.

Batch workflow definition

In the batch workflow definition:

  • the batch workflow is given a name
  • the batch value lists are defined for each input or output
  • the tasks are defined
  • input ports and output ports are defined to accept batch values if applicable.

Business Rules

  1. The number of values in the batch value lists determines the number of workflows that will be started by the batch workflow request.

  2. All inputs and outputs that accept batch values must have the same number of values in the list. If not, the batch workflow will error out.

  3. The batch value names in the batch workflow definition must be identical to the batch value names in the task definition.

For example, if the name of batch value list in the workflow definition is "input_data", the batch value name defined on the input port must also be "input_data."

This batch value list is named "input_data."

      "batch_values": [
           {
               "name": "input_data",
               "values": [
                   "CAT1",
                   "CAT2",
                   "CAT3"
               ]
           },

The list above corresponds to the value for the input port named "data." The value in double curly brackets is also named "input_data."

  "tasks": [
           {
               "name": "task_1",
               "outputs": [
                   {
                       "name": "data",
                       "persist": true,
                       "persistLocation": "{{destination}}""
                   }
               ],
               "inputs": [
                   {
                       "name": "data",
                       "value": "{{input_data}}"
                   }

Batch values

The batch workflow definition includes a section called "batch_values." In the example below, each list has three values. As a result, three workflows will be spawned and run in parallel.

Batch value lists must have the same number of values. The first value in the first input + the first value in the second input + the first value of the third input will create the first workflow.

Batch workflow request example

This is an example of a batch workflow request. In this example, there are two input ports and one output port that accept batch values. Each batch value list has three values.

{
       "name": "batch_test",
       "batch_values": [
           {
               "name": "input_data",
               "values": [
                   "CAT1",
                   "CAT2",
                   "CAT3"
               ]
           },
           {
               "name": "input_dem",
               "values": [
                   "SRTM90",
                   "SRTM120",
                   "SRTM150"
               ]
           },
           {
               "name": "destination",
               "values": [
                   "result/1",
                   "result/2",
                   "result/3"
               ]
           }
       ],
       "tasks": [
           {
               "name": "task_1",
               "outputs": [
                   {
                       "name": "data",
                       "persist": true,
                       "persistLocation": "{{destination}}""
                   }
               ],
               "inputs": [
                   {
                       "name": "data",
                       "value": "{{input_data}}"
                   },
                   {
                       "name": "demspecifier",
                       "value": "{{input_dem}}"
                   }
               ],
               "taskType": "test_task"
           }
       ]
    }

Rate Limits

Batch workflows can run up to 100 workflows concurrently. This number may vary, depending on your account limits. If too many inputs are submitted in a single batch workflow request, the system will return a "too many workflows launched" error message.

Checking the batch workflow's status

Checking a batch workflow's status will return a list of all workflows in the batch and their individual states. Although there is a batch workflow ID, there is no batch workflow state.

To check the states of the workflows in a batch workflow, see Get a batch workflow .

Canceling a Batch Workflow

Submitting a request to cancel a batch workflow will cancel all individual workflows in the batch that have not already completed. For more about canceling a batch workflow, see Cancel a batch workflow.

To cancel a single workflow in a batch, use the workflows endpoint to submit the request. See Cancel a Workflow .

Error messaging

If the value names in the workflow and task sections don't match, or If there is a batch value list in the workflow definition, but no corresponding batch value, the system will return the following error:

"batch value names do not match in the workflow JSON"

If the number of items in the input value lists don't match, the system will return the following error:

"invalid number of parameters. batch values must be consistent across all workflow."

Multiplex Ports

Multiplex ports can be used when a task has a variable number of inputs and outputs. A port is defined as "multiplex" during task definition. Then when a workflow runs the task, multiple inputs can be defined for that port.

Create a multiplex port in a task

Multiplex ports are set in the task definition. To define a task input port as a multiplex port,

  1. Give the input port a name
  2. Add "multiplex": true to the inputPort Descriptors

For example, in the task “DGLayers”, the input port named “SRC” can accept more than one input because it was defined as a multiplex port. The number of inputs that will be accepted is determined when the task is run as part of a workflow.

When the DGLayers task was defined, this port was given a single name, “SRC”, and "multiplex": true was included in the inputportDescriptor.

"inputPortDescriptors": [
  {
    "name": "SRC",
    "required": true,
    "type": "directory",
    "description": "S3 path containing input layers.",
    "multiplex": true
  },
  {
    "name": "recipe_filename",
    "required": true,
    "type": "string"
  },

Output ports can also be set up as multiplex if the number of output files will vary.

In the DGLayers example, the outputPort “DST” is a multiplex port.

"outputPortDescriptors": [
    {
      "required": true,
      "type": "directory",
      "multiplex": true,
      "name": "DST",
      "description": "Output directory."
    },

When the task is run as part of a workflow, the multiple values for the multiplex port are defined. This example shows the convention used to describe the values.

“inputs”: [
{
    “name”: “SRC_1”,
    “value”: “val_1”
},
{
    “name”: “SRC_2”,
    “value”: “val_2”
}

gbdxtools example

If you're running gbdxtools, use this example. Multiplex ports are supported like normal ports when a workflow is run.

dgl_task = gbdx.Task("DGLayers_v_2_0", SRC_1=”val_1”, SRC_2=”val_2”, SRC_3=”val_3”, recipe_dir=recipe_dir, recipe_filename=recipe_filename)
save_dgl_task = gbdx.Task("SaveToS3",data=dgl_task.outputs.DST.value,destination="s3://<your-bucket>", access_key_id="<your-access-key>", secret_key="<your-secret-key>", session_token="<your-session-token>")
workflow = gbdx.Workflow([ dgl_task,save_dgl_task ])

Note: Name/Source pairs are supported in addition to name/value pairs.

Auto Ordering Task

The auto ordering task places an order for a catalog ID. If the catalog ID has already been delivered to the catalog, the task will return its S3 location. If the catalog ID needs to be ordered, the task will place the order and wait for it to be delivered. Once the order has been delivered, the task will return the S3 location.

GBDX Registered Task Name: Auto_Ordering
Input: a single Catalog ID
Output: S3 location for the catalog ID

Task Definition

This is the task definition for the auto ordering task.

{
    "inputPortDescriptors": [
        {
            "required": true,
            "description": "Catalog Id",
            "name": "cat_id",
            "type": "string"
        }
    ],
    "outputPortDescriptors": [
        {
            "name": "s3_location",
            "type": "string"
        }
    ],
    "containerDescriptors": [
        {
            "type": "DOCKER",
            "command": "",
            "properties": {
                "image": "tdgp/auto_ordering",
                "mounts": [
                    {
                        "local": "$task_data_dir",
                        "container": "/mnt/work",
                        "read_only": false
                    }
                ]
            }
        }
    ],
    "description": "GBDX Auto Ordering Task",
    "name": "Auto_Ordering",
    "properties": {
        "authorizationRequired": true,
        "timeout": 36000,
        "isPublic": true
    }
}

How to Use the Auto Ordering Task

Submit the following Workflow to the workflows/v1/workflows endpoint with a valid Cat_ID:

    {
        "name": "Auto_Ordering_Workflow",
        "tasks": [
            {
                "outputs": [
                    {
                        "name": "s3_location"
                    }
                ],
                "name": "auto_ordering",
                "taskType": "Auto_Ordering",
                "impersonation_allowed": true,
                "inputs": [
                    {
                        "name": "cat_id",
                        "value": "<string>"
                    }
                ]
            }
        ]
    }

Rules for the Auto Ordering Task

The Auto Ordering task will submit an order using your GBDX credentials to the Ordering Endpoint orders/v2/ordercb The "ordercb" endpoint returns a callback when the order is delivered.

  • If the order has already been delivered, the task will return the S3 location immediately.

  • If the order has not been delivered, the task will wait for the order to be fulfilled.

  • Once the order is delivered,a callback is sent. When the callback is received, the Auto Ordering task will resume and return the S3 Location.

  • When the Auto Ordering task is in the "waiting" state, it will return the order ID as a "note." You can use this order ID to query the Orders API for status. See Get Order Status v2.

  • The S3 Location returned from the Auto Ordering task can be piped into another task by using the output port s3_location

Workflow Events for the Auto Ordering Task

The following are workflow events for the Auto Ordering Task:

        "Events": [
            {
                "task": "auto_ordering",
                "timestamp": "2016-09-01T19:15:42.795775+00:00",
                "when": "20 minutes ago",
                "note": "",
                "state": "pending",
                "event": "submitted"
            },
            {
                "task": "auto_ordering",
                "timestamp": "2016-09-01T19:16:45.145101+00:00",
                "when": "19 minutes ago",
                "note": "instance_id: i-86f35cb7",
                "state": "pending",
                "event": "scheduled"
            },
            {
                "task": "auto_ordering",
                "timestamp": "2016-09-01T19:16:45.225323+00:00",
                "when": "19 minutes ago",
                "note": "instance_id: i-14933a25, domain: default",
                "state": "running",
                "event": "started"
            },
            {
                "task": "auto_ordering",
                "timestamp": "2016-09-01T19:17:40.404544+00:00",
                "when": "18 minutes ago",
                "note": "instance_id: i-14933a25,  Note: Waiting for Ordering System to Complete Order. OrderId: b77224ec-3015-4460-89f9-17ddaba9a6bc",
                "state": "pending",
                "event": "waiting"
            },
            {
                "task": "auto_ordering",
                "timestamp": "2016-09-01T19:33:57.300772+00:00",
                "when": "2 minutes ago",
                "note": "instance_id: i-86f35cb7",
                "state": "pending",
                "event": "scheduled"
            },
            {
                "task": "auto_ordering",
                "timestamp": "2016-09-01T19:33:57.417425+00:00",
                "when": "2 minutes ago",
                "note": "instance_id: i-959039a4, domain: default",
                "state": "running",
                "event": "started"
            },
            {
                "task": "auto_ordering",
                "timestamp": "2016-09-01T19:35:23.740966+00:00",
                "when": "seconds ago",
                "note": "instance_id: i-959039a4,  Note: Waiting for Ordering System to Complete Order. OrderId: b77224ec-3015-4460-89f9-17ddaba9a6bc, Order has been Completed, Location: s3://receiving-dgcs-tdgplatform-com/055567707010_01_003",
                "state": "complete",
                "event": "succeeded"
            }
        ]
    }

Workflow and Task Callbacks

Workflows and tasks within a workflow can have a URL callback defined. The callback is an HTTP POST to the URL that is defined in the task or workflow. It includes a small JSON packet describing the event that triggered the callback.

Workflow callbacks are set as part of the workflow definition.

Task callbacks are only set at the "run" time of a task, not definition time. That means you'll include it in the task definition within the workflow, not in the task definition in the task registry.

Task Callbacks

To set a callback on a Task, add the following to the Workflow JSON that is passed when starting a workflow with a set of tasks:

    {
      "name": "AOP_Strip_Processor_protogenV2LULC",
      "callback": "http://www.somehost.tld/some/url",
      "tasks": [
        {
          "name": "AOP",
          "outputs": [
            {
              "name": "log"
            }
          ],
          "inputs": [
            {
              "name": "data",
              "value": "<string>"
            }
          ],
          "taskType": "AOP_Strip_Processor",
          callback": "http://www.somehost.tld/someother/url"
        }
      ]
    }

Note that while the example shows two different URL end points, that is not a requirement. However the data that is passed to the endpoint will be different depending on whether the callback is from a task or from a workflow. The message that is sent when a task changes state is below. This is the same message structure for all event states.

  {
      "userName": "testuser",
      "payload": {
        "taskDomain": "default",
        "instanceSize": "LOCAL",
        "workflowName": "AOP_Strip_Processor_protogenV2LULC",
        "instanceId": "LOCAL",
        "taskState": "succeeded",
        "workflowId": "4517305590090108113",
        "workflowState": "running",
        "taskId": "4517305590044912472",
        "taskType": "AOP_Strip_Processor",
        "taskName": "Automated LULC"
      },
      "entity": "task",
      "environment": "alpha",
      "source": "worker",
      "action": "state changed",
      "properties": {
      },
      "accountId": "<YOUR ACCOUNT ID>"
    }

A Workflow event message is similar but has fewer fields:

{
      "message": {
        "userName": "testuser",
        "payload": {
          "workflowId": "4517315362701210938",
          "workflowName": "AOP_Strip_Processor_protogenV2LULC",
          "workflowState": "succeeded"
        },
        "entity": "workflow",
        "environment": "LOCAL",
        "source": "decider",
        "action": "state changed",
        "properties": {

        },
        "accountId": "<YOUR ACCOUNT ID>"
      }
    }

Additional Information

The URL that is called with the JSON message can be any publicly accessible URL. For example, you won't be able to test with "localhost" or anything that can't be accessed from the GBDX environment.

The callback manager will wait for five seconds to connect with the URL endpoint and an additional five seconds for the endpoint to respond. The endpoint must respond with an HTTP "200" code from the callback. If it responds with anything else or if the call times out, the system will try again in 30 seconds. It will try this 5 times before giving up.

The current configuration accepts any SSL certificates. You can use a self-signed certificate for the callback URL.