OMII-UK Workflow Monitor Web Application - Users Guide

John S Robinson - J.S.Robinson@soton.ac.uk

Stian Soiland-Reyes - ssoiland@cs.man.ac.uk


http://www.omii.ac.uk


Created: 29 April 2008

Modified: 1 May 2008



This application performs periodic monitoring of workflow Activities (WSDL, Soaplab, Biomoby and Biomart are currently supported) and provides information on their current and historical status. The principal interface is a RESTful web service, employing HTTP POST / GET / PUT / DELETE methods to Create / Read / Update / Delete resources.

Main resources

A User has a collection of Workflows, and a collection of Monitors.

A Workflow originates as Scufl file (as produced by Taverna). It has 1 or more Activities

A Monitor is a "job" of what is to be tested when and how. A Monitor has one Workflow and 0 or more Reports. (i.e. reports of testing that workflow). A Monitor has a Status, which is basically the result of the latest Report.

A Report describes how the execution of a Monitor ended up. If appropriate, saying which Activities caused it to fail.

An Activity is an invokable service as described by Taverna 2, for instance a WSDL activity. They are the targets of error reports in a HealthReport.

A HealthReport is generated each time a an Activity is tested.

URL Layout



URL template

Resource

/

Capabilities


/user;current

Current user (Redirects to /users/{user} based on auth info)

/user/{user}

User

/user/{user}/workflows

Collection of Workflows

/user/{user}/workflow/{workflow}

Workflow (Scufl file)

/user/{user}/workflow/{workflow}/activities

Collection of Activities

/user/{user}/workflow/{workflow}/monitors

Collection of Monitors

/user/{user}/monitors

Collection of Monitors

/user/{user}/monitor/{monitor}

Monitor

/user/{user}/monitor/{monitor}/reports

Collection of Reports

/user/{user}/monitor/{monitor}/reports.atom

ATOM feed

/user/{user}/monitor/{monitor}/report/{report}

Report

/user/{user}/monitor/{monitor}/status

Status

/activities

Collection of all known Activities (hidden for unprivileged users)

/activity/{activity}

Activity

/activity/{activity}/workflows

Collection of Workflows

/activity/{activity}/healthreports

Collection of HealthReports



Example of usage and representation documents

Getting started: The capabilities document

GET / HTTP/1.1

HTTP/1.1 200 OK
Content-Type: application/vnd.omii.tam+xml
<capabilities xmlns="http://www.omii.ac.uk/2008/workflowmonitor" xmlns:xlink="http://www.w3.org/1999/xlink">
  <currentUser xlink:href="http://host:8080/WorkflowMonitor/Restlet/user;current" />
  <activities xlink:href="http://host:8080/WorkflowMonitor/Restlet/activities" />
</capabilities>

This is the starting point for the REST client, the Capabilities document. Everything else is based on understanding the resource representations, following its links and using the 4 basic CRUD operations of HTTP: POST (create), GET (read), PUT (update), DELETE (delete).

The user document

GET http://host:8080/user;current HTTP/1.1

HTTP/1.1 307 Temporary redirect
Location: http://host:8080/user/foo

GET http://host:8080/user/foo HTTP/1.1

HTTP/1.1 200 OK
Content-Type: application/vnd.omii.tam+xml

<capabilities xmlns="http://www.omii.ac.uk/2008/workflowmonitor" xmlns:xlink="http://www.w3.org/1999/xlink">
  <currentUser xlink:href="http://host:8080/WorkflowMonitor/Restlet/user;current" />
  <activities xlink:href="http://host:8080/WorkflowMonitor/Restlet/activities" />
</capabilities>

Uploading a workflow

Upload a workflow by POSTing a Scufl document:

POST http://host:8080/WorkflowMonitor/Restlet/user/foo/workflows HTTP/1.1
Content-Type: application/vnd.taverna.scufl+xml

HTTP/1.1 201 Created
Location: http://host:8080/WorkflowMonitor/Restlet/user/foo/workflow/c262857a-d43d-4dfe-b298-3f61210478cd

Confirm success by retrieving the Scufl document:

GET http://host:8080/WorkflowMonitor/Restlet/user/foo/workflow/c262857a-d43d-4dfe-b298-3f61210478cd HTTP/1.1
Accept: application/vnd.taverna.scufl+xml

HTTP/1.1 200 OK
Content-Type:  application/vnd.taverna.scufl+xml

<?xml version="1.0" encoding="UTF-8"?>
<s:scufl xmlns:s="http://org.embl.ebi.escience/xscufl/0.1alpha" version="0.2" log="0">
  <s:workflowdescription       (...)
</s:scufl>

A request without specifying the Taverna SCUFL content type gets a list of what the service knows about this workflow:

GET http://host:8080/WorkflowMonitor/Restlet/user/foo/workflow/c262857a-d43d-4dfe-b298-3f61210478cd HTTP/1.1

HTTP/1.1 200 OK
Content-Type: application/vnd.omii.tam+xml

<workflow xmlns="http://www.omii.ac.uk/2008/workflowmonitor" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:dc="http://dublincore.org/documents/dcmi-namespace/">
  <dc:title>testWorkflow</dc:title>
  <dc:author>zucc</dc:author>
  <url xlink:href="http://host:8080/WorkflowMonitor/Restlet/user/foo/workflow/c262857a-d43d-4dfe-b298-3f61210478cd" />
  <activities xlink:href="http://host:8080/WorkflowMonitor/Restlet/user/foo/workflow/c262857a-d43d-4dfe-b298-3f61210478cd/activities" />
  <monitors xlink:href="http://host:8080/WorkflowMonitor/Restlet/user/foo/workflow/c262857a-d43d-4dfe-b298-3f61210478cd/monitors" />
</workflow>

Following <activities> gives a list of all the activities recognised in this workflow. Following <monitors> gives a list of all the monitors (this user has access to) that test this workflow.

Adding and retrieving a monitor

A monitor is added by POSTing a Monitor document:

POST /user/foo/monitors HTTP/1.1
Content-Type: application/vnd.omii.tam+xml

<?xml version="1.0" encoding="UTF-8"?>
	<monitor xmlns="http://www.omii.ac.uk/2008/workflowmonitor" xmlns:xlink="http://www.w3.org/1999/xlink">
			<monitorable xlink:href="http://host:8080/WorkflowMonitor/Restlet/user/foo/workflow/c262857a-d43d-4dfe-b298-3f61210478cd/>
	</monitor>

HTTP/1.1 201 Created
Location: http://host:8080/WorkflowMonitor/Restlet/user/foo/monitor/48187418-4511-43ed-95c3-3f4113aebfb7

The Location header gives the URI for the newly created monitor. Retrieving the Monitor document shows the defaults provided by the server, in addition to links to more information about the monitor:

GET http://host:8080/WorkflowMonitor/Restlet/user/foo/monitor/48187418-4511-43ed-95c3-3f4113aebfb7 HTTP/1.1

HTTP/1.1 200 OK
Content-Type: application/vnd.omii.tam+xml

<monitor xmlns="http://www.omii.ac.uk/2008/workflowmonitor" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:dc="http://dublincore.org/documents/dcmi-namespace/">
  <dc:title />
  <monitorable xlink:href="http://host:8080/WorkflowMonitor/Restlet/user/foo/workflow/c262857a-d43d-4dfe-b298-3f61210478cd" />
  <updateInterval>PT15M</updateInterval>
  <reports xlink:href="http://host:8080/WorkflowMonitor/Restlet/user/foo/monitor/48187418-4511-43ed-95c3-3f4113aebfb7/reports" />
  <status xlink:href="http://host:8080/WorkflowMonitor/Restlet/user/foo/monitor/48187418-4511-43ed-95c3-3f4113aebfb7/status" />
</monitor>

The Monitor can be modified and PUT back, for example giving it a title and changing the updateInterval:

PUT http://host:8080/WorkflowMonitor/Restlet/user/foo/monitor/48187418-4511-43ed-95c3-3f4113aebfb7 HTTP/1.1
Content-Type: application/vnd.omii.tam+xml

<monitor xmlns="http://www.omii.ac.uk/2008/workflowmonitor" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:dc="http://dublincore.org/documents/dcmi-namespace/">
  <dc:title>Foo's monitor</dc:title>
  <monitorable xlink:href="http://host:8080/WorkflowMonitor/Restlet/user/foo/workflow/c262857a-d43d-4dfe-b298-3f61210478cd" />
  <updateInterval>PT5M</updateInterval>
  <reports xlink:href="http://host:8080/WorkflowMonitor/Restlet/user/foo/monitor/48187418-4511-43ed-95c3-3f4113aebfb7/reports" />
  <status xlink:href="http://host:8080/WorkflowMonitor/Restlet/user/foo/monitor/48187418-4511-43ed-95c3-3f4113aebfb7/status" />
</monitor>

HTTP/1.1 204 No content

The user's collection of Monitors can be retrieved, including the previously created monitor with the updated title:

GET http://host:8080/WorkflowMonitor/Restlet/user/foo/monitors HTTP/1.1

HTTP/1.1 200 OK
Content-Type:  application/vnd.omii.tam+xml

<monitors xmlns="http://www.omii.ac.uk/2008/workflowmonitor">
  <monitor xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:dc="http://dublincore.org/documents/dcmi-namespace/">
    <dc:title>Foo's monitor</dc:title>
    <monitorable xlink:href="http://host:8080/WorkflowMonitor/Restlet/user/foo/workflow/c262857a-d43d-4dfe-b298-3f61210478cd" />
    <updateInterval>PT5M</updateInterval>
    <reports xlink:href="http://host:8080/WorkflowMonitor/Restlet/user/foo/monitor/48187418-4511-43ed-95c3-3f4113aebfb7/reports" />
    <status xlink:href="http://host:8080/WorkflowMonitor/Restlet/user/foo/monitor/48187418-4511-43ed-95c3-3f4113aebfb7/status" />
  </monitor>
</monitors>




Checking Reports

GET http://host:8080/WorkflowMonitor/Restlet/user/foo/monitor/48187418-4511-43ed-95c3-3f4113aebfb7/reports HTTP/1.1

HTTP/1.1 200 OK
Content-Type:  application/vnd.omii.tam+xml

<reports xmlns="http://www.omii.ac.uk/2008/workflowmonitor" xmlns:xlink="http://www.w3.org/1999/xlink">
  <monitor xlink:href="http://host:8080/WorkflowMonitor/Restlet/user/foo/monitor/48187418-4511-43ed-95c3-3f4113aebfb7" />
  <feed xlink:href="http://host:8080/WorkflowMonitor/Restlet/user/foo/monitor/48187418-4511-43ed-95c3-3f4113aebfb7/reports.atom" />
  <report xlink:href="http://host:8080/WorkflowMonitor/Restlet/user/foo/monitor/48187418-4511-43ed-95c3-3f4113aebfb7/report/20080430T145106+0100" />
  <report xlink:href="http://host:8080/WorkflowMonitor/Restlet/user/foo/monitor/48187418-4511-43ed-95c3-3f4113aebfb7/report/20080430T144228+0100" />
</reports>

An Atom feed is also available, the URL is given in the <feed> element above:

GET http://host:8080/WorkflowMonitor/Restlet/user/foo/monitor/48187418-4511-43ed-95c3-3f4113aebfb7/reports.atom HTTP/1.1
Accept: application/atom+xml

HTTP/1.1 200 OK
Content-Type: application/atom+xml

<feed xmlns="http://www.w3.org/2005/Atom">
  <title>Reports for Monitor - Foomon -</title>
  <link href="http://host:8080/WorkflowMonitor/Restlet/user/foo/monitor/48187418-4511-43ed-95c3-3f4113aebfb7/reports.atom" rel="self" />
  <link href="http://host:8080/WorkflowMonitor/Restlet/user/foo/monitor/48187418-4511-43ed-95c3-3f4113aebfb7/reports" />
  <updated>2008-04-30T14:51:06+01:00</updated>
  <id>http://host:8080/WorkflowMonitor/Restlet/user/foo/monitor/48187418-4511-43ed-95c3-3f4113aebfb7/reports</id>
  <author>
    <name>Workflow Monitor</name>
    <uri>http://host:8080/WorkflowMonitor/Restlet/user/foo/monitor/48187418-4511-43ed-95c3-3f4113aebfb7</uri>
  </author>
  <entry>
    <title>Successful at 2008-04-30T14:51:06+0100</title>
    <link href="http://host:8080/WorkflowMonitor/Restlet/user/foo/monitor/48187418-4511-43ed-95c3-3f4113aebfb7/report/20080430T145106+0100" />
    <id>http://host:8080/WorkflowMonitor/Restlet/user/foo/monitor/48187418-4511-43ed-95c3-3f4113aebfb7/report/20080430T145106+0100</id>
    <updated>2008-04-30T14:51:06+01:00</updated>
    <summary>All activities succeeded : Reliability - 1.0</summary>
  </entry>
  <entry>
    <title>Successful at 2008-04-30T14:42:28+0100</title>
    <link href="http://host:8080/WorkflowMonitor/Restlet/user/foo/monitor/48187418-4511-43ed-95c3-3f4113aebfb7/report/20080430T144228+0100" />
    <id>http://host:8080/WorkflowMonitor/Restlet/user/foo/monitor/48187418-4511-43ed-95c3-3f4113aebfb7/report/20080430T144228+0100</id>
    <updated>2008-04-30T14:42:28+01:00</updated>
    <summary>All activities succeeded : Reliability - NaN</summary>
  </entry>
</feed>

To retrieve an individual report:

GET http://host:8080/WorkflowMonitor/Restlet/user/foo/monitor/48187418-4511-43ed-95c3-3f4113aebfb7/report/20080430T145106+0100 HTTP/1.1

HTTP/1.1 200 OK
Content-Type:  application/vnd.omii.tam+xml

<report xmlns="http://www.omii.ac.uk/2008/workflowmonitor" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:dc="http://dublincore.org/documents/dcmi-namespace/">
  <dc:created>2008-04-30T14:51:06+0100</dc:created>
  <monitor xlink:href="http://host:8080/WorkflowMonitor/Restlet/user/foo/monitor/48187418-4511-43ed-95c3-3f4113aebfb7" />
  <monitorable xlink:href="http://host:8080/WorkflowMonitor/Restlet/user/foo/workflow/c262857a-d43d-4dfe-b298-3f61210478cd" />
  <state>OK</state>
  <message>All activities succeeded</message>
  <loggedHealthReports>
    <healthReport>
      <dc:modified>2008-04-30T14:50:07+0100</dc:modified>
      <activity xlink:href="http://host:8080/WorkflowMonitor/Restlet/activity/07c614c4-8e6e-4037-97aa-1450f6b2ea8b" />
      <state>0</state>
      <message>The endpoint [http://www.ebi.ac.uk/soaplab/emboss4/services/edit.seqret] responded with a response code of 200</message>
    </healthReport>
    <healthReport>
      <dc:modified>2008-04-30T14:50:07+0100</dc:modified>
      <activity xlink:href="http://host:8080/WorkflowMonitor/Restlet/activity/ffb1e1bb-a0fc-4f42-a936-4e97ab4f146a" />
      <state>0</state>
      <message>The WSDL [http://bladevm9.omii.ac.uk:8080/axis/services/ComedyWs?wsdl] responded OK; The endpoint [http://bladevm9.omii.ac.uk:8080/axis/services/ComedyWs] responded, with a response code of 200; rpc/encoded is OK;</message>
    </healthReport>
    <healthReport>
      <dc:modified>2008-04-30T14:50:07+0100</dc:modified>
      <activity xlink:href="http://host:8080/WorkflowMonitor/Restlet/activity/f636f6ae-8854-4cfa-bace-b9a09185dfa1" />
      <state>0</state>
      <message>The WSDL [http://bladevm9.omii.ac.uk:8080/axis/services/ComedyWs?wsdl] responded OK; The endpoint [http://bladevm9.omii.ac.uk:8080/axis/services/ComedyWs] responded, with a response code of 200; rpc/encoded is OK;</message>
    </healthReport>
  </loggedHealthReports>
</report>

Status

The status gives information about the current state of the workflow/activity:

GET http://host:8080/WorkflowMonitor/Restlet/user/foo/monitor/48187418-4511-43ed-95c3-3f4113aebfb7/status HTTP/1.1

HTTP/1.1 200 OK
Content-Type:  application/vnd.omii.tam+xml

<status xmlns="http://www.omii.ac.uk/2008/workflowmonitor" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:dc="http://dublincore.org/documents/dcmi-namespace/">
  <dc:modified>2008-04-30T14:51:45+0100</dc:modified>
  <monitor xlink:href="http://host:8080/WorkflowMonitor/Restlet/user/foo/monitor/48187418-4511-43ed-95c3-3f4113aebfb7" />
  <state>OK</state>
  <reliability>1.0</reliability>
</status>

The reliability is the number of successful reports / the total number of reports.

Activities

Activities are created by registering a Scufl workflow, they cannot be registered independently. However, data about an Activity can be retrieved:

GET http://host:8080/WorkflowMonitor/Restlet/activity/07c614c4-8e6e-4037-97aa-1450f6b2ea8b HTTP/1.1

HTTP/1.1 200 OK
Content-Type:  application/vnd.omii.tam+xml

<activity xmlns="http://www.omii.ac.uk/2008/workflowmonitor" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:dc="http://dublincore.org/documents/dcmi-namespace/">
  <dc:title>Endpoint: http://www.ebi.ac.uk/soaplab/emboss4/services/edit.seqret Polling interval:0 Polling backoff: 1.0 PollingIntervalMax: 0</dc:title>
  <activityType>Soaplab</activityType>
  <workflows xlink:href="http://host:8080/WorkflowMonitor/Restlet/activity/07c614c4-8e6e-4037-97aa-1450f6b2ea8b/workflows" />
  <healthReports xlink:href="http://host:8080/WorkflowMonitor/Restlet/activity/07c614c4-8e6e-4037-97aa-1450f6b2ea8b/healthreports" />
</activity>

Following the <workflows> and <healthReports> links will return a list of Workflows using the activity and a list of HealthReports for that activity.