Link Search Menu Expand Document

Workflow Repository

Introduction

Workflows combine multiple complex flows into a single, human understandable logical representation of a business process.

A workflow is very similar to a BPMN process. It contains action steps – which are flow steps – and logical gateways. While individual flows represent a very specific sub-process, where the main actors are machines, workflows bring together with logical gateways more complex and overarching processing embedding asynchronous user interaction.

Workflow

API Reference Implementation

Technologies used

  • Node.js
  • MongoDB

How it works

A workflow consist of 1-n consecutive flows. This service defines which flows make up a workflow and how and when these should be executed by the orchestrator. Workflows can also be either static (as a template) but also freely extensible.

Workflow

One of the challenges is to pass data between flows. Assuming, that two flows are part of a workflow and are processed sequentially, the second flow may rely on data generated by the first flow, especially if a logical gateway is involved. This shall be achieved by passing the snapshot data (see Snapshot Repository) from all previous flows to the subsequent flows. This also implies, that snapshot data will have a special flag, indicating, that it should only be deleted, when the corresponding workflow is stopped/deleted. Similarly to flows, workflows also have different lifecycle states (e.g. running, stopped)

The complex structure of workflows (again, imagine a complex BMPN diagram) requires logical components where two or more flows are either started in parallel or are joined together into a new flow later. There can also be conditional execution of only one branch (conditional switches). These logical operations are predefined by the user and executed in special logical gateway components, which will be parts of a flow. See the “Rules Engine” documentation for more info.

Currently, the Orchestrator service receives the response from each flow step, determines the next step(s) and executes it/them accordingly. If a conditional switch (logic gateway) is placed within a single flow (in contrast to the last node of a previous flow), then this logic gateway needs to instruct the Orchestrator to execute only a specific stepId and not all subsequent nodes where multiple edges lead to multiple nodes from this logical gateway node. This also means, that the Orchestrator must evaluate the received payload.

Async operations

Manual steps or asynchronous operations generally in a process/workflow require special triggers. The existing Scheduler and Webhook triggers can be used to start a specific flow. There are however cases, where a flow step creates an entity (t0), and the next step t1 should only execute when a specific condition for the said entity is met, e.g. t0 creates a task and t1 should execute when this task is complete. When dealing with external systems that do not support webhooks, a scheduled polling would the alternative.

With the workflows, the Scheduler and Orchestrator services shall be modified to support this. As visualized in the diagram, the logical gateway checks for the condition (in this case – the task is complete) and if not, it instructs the scheduler to execute a specific step in the current flow rather the restart a flow.

The operation described above may be a single, non-recurring step. The scheduler functionality must be extended to support a one-time execution of flowId + stepId.

Workflow

Passing data between flows

As mentioned previously, snapshot data may be required by multiple flows. Therefore, the Ferryman library will receive a unique identifier (workflow key) for the snapshot. For each component in a flow, Ferryman should fetch the Snapshot data and store it on step completion. This feature is currently only relevant for workflows and should be optional.

As each step knows it’s id (step_id), the data can be stored either as a flat object structure, or an array with step_id property in each array object.

Workflow

Versioning

When a workflow is started, it shall create an immutable copy (a snapshot). All flows referenced by this workflow will also be started automatically, making the immutable as well.

Use Cases and requirements in Open Integration Hub

OIH should provide SMEs (small and medium enterprises) a way to build and run business processes in a convinient manner. Workflows are ment to do exactly this.

Workflow structure

{
  "_id": "59f9f2ba112f28001921f274",
  "tenant": "59f9f2ba112f28001921f245",
  "type": "STATIC",
  "name": "My Workflow",
  "flows": [
    {
        "flowId": "59feb5ba112f28001921f233",
        "successor": "59feb5ba112f28001921f234"
    },
    {
      "flowId": "59feb5ba112f28001921f234"
    }
  ],
  "meta":{}
}