Drag-and-drop Compute Job Submissions

Google Summer of Code 2008 ideas

Primary Mentor: Dr Jano van Hemert, j.vanhemert at ed.ac.uk (jvhemert at gmail.com).
Secondary Mentor: Dr Jos Koetsier, jos at nesc.ac.uk.

Background

The research group at the National e-Science Centre of the University of Edinburgh is part of the School of Informatics, which has the highest rating of computer science research in the UK. We continuously have a set of research students (undergraduate year 4, MSc and PhD) interacting closely with the academic staff. A list of past, current, and available projects is on the research web site. As a multi-disciplinary centre, students will benefit from interaction with many other scientific domains. Also, the centre includes the e-Science Institute, which brings in many international visitors, workshops and conferences, and a training team that can assist with questions about grid computing.

Project Goals

To build an extension on an existing user-mountable GridDrive that allows users to start compute jobs on remote computers using drag-and- drop.

Project Requirements

This project requires a person with a hacker personality in the positive sense, with much experience with Linux and its surrounding culture. Proficient experience with using C libraries is a must.

Project Details

To enhance the experience of users that want to use Grid Computing Infrastructures or High-Performance Computing Facilities, several attempts have been made to get away from the command-line approach to job schedulers. Many of these centre around graphical front-ends, either delivered via a browser or as a stand-alone application. Some have experimented using a drag-and-drop interface, but these have often included many complications for the users, such as requiring a specific job-description file to be written.

We already have working a user-mountable drive, based on an extension of ssh to allow proper authentication via certificates and on FUSE (Open Source software to allow users to mount drives and other devices in user space rather than kernel space). The aim here is to make submitting jobs as easy as a drag-and-drop of an input file on a mounted executable.

This would allow very cool things to happen, which are currently impossible because of the way the submission of compute jobs is traditionally handled. By directly mounting the disk and talking directly to the applications on the server we can start streaming data in and out of programs. Also, we can easily set up pipelines, such as common under any Unix environment, but where some of the applications used are running remotely.

The main benefits or this approach are. It would allow users to use computing infrastructure only where appropriate in their pipelines and scripts. In all other circumstances, users will work on their system with all their preferred applications, libraries and other peculiarities. As the drivers you will develop are running in user space, any user can install it on their system. The whole system is transparent to the user, let's be honest, what do you prefer, dragging files on to a USB stick, or use some specifically designed piece of software to handle file transfers?