Service Collaboration
This document is designed to describe different service collaboration examples. It acts as a starting point to easily understanding the architecture of the Open Integration Hub. Most of the examples are triggered by user interactions (e.g. starting a flow) and only the “happy path” i.e. success scenario is described.
Each example is described through a graphical overview, a textual description, and pre-conditions. For further information for a specific version please have a look at the services.
- Starting a flow
- Execute Polling Flow
- Execute Webhook Flow
- Request Resources
- Creating audit log records
Starting a flow
Pre-Conditions: None.
This example describes the scenario of starting a flow. Once the user starts a flow the following steps are processed:
- The client starts a flow using the flow repository’s REST API.
Flow Repository
sets the flow’sstatus
tostarting
and raises the eventflow.starting
.- There are 3 services listening to the event
flow.starting
: Webhooks, Scheduler, and Component Orchestrator.Webhooks
andScheduler
examine the event’s payload and decide if they need to react appropriately. We will discuss the exact reaction of both services later in this document. - Upon receiving
flow.starting
event theComponent Orchestrator
starts deploying local components. Once all local components were deployed,Component Orchestrator
raises theflow.started
event – Keep in mind that global components need to be started manually. Flow Repository
receives theflow.started
event and switches flow’sstatus
property fromstarting
tostarted
.Webhooks
receives theflow.started
event and starts receiving incoming HTTP calls for the given flow.Scheduler
receives theflow.started
event and starts scheduling the flow, according to its cron property.- When a client stops a running flow using the flow repository’s REST API, the event
flow.stopping
is raised which is causing an inverse reaction chain of events.
Now let’s discuss the individual services in detail:
Flow repository
POST /flows/{id}/start
: Used to start a flowPOST /flows/{id}/stop
: Used to stop a flow
Upon receiving the HTTP call for starting a flow, Flow Repository sets the flows status
to starting
. Upon receiving a stopping request it sets the status to stopping
. If the flow has been started and the flow repository receives flow.started
event, it sets the status to active
in contrast, it sets it to inactive
upon receiving flow.stopped
. The schema of the event payload is shown below.
Event:
type: object
required:
- headers
- payload
properties:
headers:
type: object
required:
- name
- createdAt
- serviceName
properties:
name:
type: string
createdAt:
type: string
format: date-time
serviceName:
type: string
payload:
type: object
The payload
property is an arbitrary object to be sent with the event. Flow repository will send the entire flow as payload
.
Webhooks
Upon receiving flow.starting
event the service checks if the cron
property is not set. If so, the service persists a data record in his local DB but doesn’t start receiving HTTP requests for the given flow yet. After receiving the flow.started
event, the service starts accepting incoming messages from the flow’s webhook URL and instructs Component Orchestrator
by publishing the flow.executed
event as soon as a valid webhook call is carried out.
Please note that the webhooks service ignores the event if the following condition is met:
cron
property is set in the event
Upon receiving the flow.stopping
event, the service deletes the record for the given flow and stops accepting requests.
Scheduler
Upon receiving flow.starting
event the service checks if the cron
property is set. If so, the service persist a data record in his local DB, but doesn’t start scheduling the given flow yet. The following table demonstrates an example of such records.
flowId | cron | dueExecution |
---|---|---|
58b41f5da9ee9d0018194bf3 | */3 * * * * | 2019-01-25T13:39:28.172 |
5b62c91afd98ea00112d5404 | 15 14 * * 1-5 | 2019-01-27T14:15:00.00 |
Upon receiving the flow.started
event the service starts scheduling the flow executions by retrieving the flow data from its local DB and instructing Component Orchestrator
by publishing the flow.executed
event.
Please note that the scheduler service ignores the event if the following condition is met:
cron
property is not set in the event
Upon receiving the flow.stopping
event, the service deletes the record for the given flow and stops scheduling flow executions.
Execute Polling Flow
Pre-Conditions: Starting a flow.
As described in scheduler section when a flow is started the service starts scheduling the flow executions. Once the scheduler finds a flow that is ready for execution it publishes flow.executed
.
Figure: executePollingFlow
Execute Webhook Flow
POST Request
Pre-Conditions: Starting a flow.
Once Webhooks receives a POST request it publishes flow.executed
. In contrast to the GET request, this request already includes the payload.
Figure: executeWebhookFlowPost
GET Request
Pre-Conditions: Starting a flow.
Once Webhooks receives a GET request it takes the url parameters and request headers and put it into the message. This means in particular the headers go to headers
while query string parameters go to body
.
An examplary webhook GET request could look like the following: GET /hook/<flow-id>?param1=value¶m2=value
Figure: executeWebhookFlowGet
Request Resources
The following example shows how a user can request a resource using IAM. The graphic below shows how this example would look like if a user request a resource from the flow repository.
- User logs in into IAM.
- IAM responds with an ephemeral token.
- User uses the ephemeral token to request a cetrain resource (e.g. a specific flow by id).
- Flow repository introspects the ephemeral token at IAM (services accounts receive a permanent token when they first register) using IAM utils (middleware).
- IAM responds with user information such as username, tenant, tenant specific role and user permissions related to this token.
- Flow Repsitory checks if the user has the permission to request the resource.
- Flow repository responds with the requested information.
Illustration of this process: (Figur requestResourceSuccess).
Figure: requestResourceSuccess
1: Ephemeral token
2: Service makes request with service account token
3: User information e.g.: username, tenant, tenant specific role, permissions
Creating audit log records
To create a record that should be stored in the audit log a service simply has to put a message onto the queue with a predefined topic. Each service decides on its own, which events should be stored in the audit log service. Audit log listens to all events having audit.*
as topic.