Open source development tools

Attendees

• Pascal Ekin – The University of Manchester
• Mike Jackson – The University of Edinburgh (scribe)
• Phil Kershaw – NDG, RAL
• Eduardo Pignotti – The University of Aberdeen

Overview

This session discussed open source tools and infrastructures, focusing on ideas for OGSA-DAI and open development.

Repository

The OGSA-DAI repository is exposed publicly, but with a private area with third-party JARs. It is hard to build and is not structured to readily support OGSA-DAI’s core and extension vision

CVS is used but not branches. GT and Taverna use both a trunk and branch, one for development and one for the most recent release and bug fixes.

SVN is recommended. It is good for releases. Developers can copy the whole of the trunk to a release area. It is very powerful. There are Eclipse plug-ins e.g. SubClipse, and tools to move from CVS to SVN.

Build

OGSA-DAI uses ANT build files and properties. ANT scripts invokes other ANT scripts - this is used to get or build dependencies, for example. There is a master build file to build all the releases.

Maven could be used to compile, test, package and publish. It allows dependency expression and can pull in JARs from off-site repositories. Nightly builds e.g. JARs can be published. Use of a local repository avoids online pull-down of JARs.

One advantage of the Maven model is that it avoids bundling. There's a smaller footprint for download. But one still has the local repository footprint.

Taverna use their custom version, Raven, but will move back to Maven.

Using Maven + ANT could be very messy. Also Maven does a lot of things behind the scenes so it can be hard to understand if things goes wrong, whereas ANT is at least visible.

OMII SDK example use HTTP GETs in Perl to pull down off-site packages.

It would be desirable to allow an open source community submit Maven configs, ANT scripts whatever e.g. to build OGSA-DAI versions with appllicatuion/community-specifc extension

Easy_install also pulls down packages (eggs) based on a list of dependencies.

ZC.BuildOut builds on that to allow versions and combinations of these eggs for consistent releases.

Test framework

The OGSA-DAI test framework builds OGSA-DAI releases, deploys these, compile and runs tests and publishes the results online. It is written in ANT with ANT extensions for iteration and try-catch blocks. It is messy and hard to extend (even for seasoned OGSA-DAI developers).

Would be useful to find out if Maven could help, what XP approaches and tools there are and to look at Jython (Python). Jython allows Java-style scripting – it's clearer than Perl and more aligned with existing OGSA-DAI developer expertise. It can call out to ANT.

Other test tools include NUnit and SQL Alchemy may be of interest.

Open source project

Governance needs to address what happens if an extension pack needs a change to the core and who is responsible for what e.g. bugs, support, licence violations, contributions policy and branding.

TRAC is a usefu project tool. It integrates with SVN and can integrate with CVS. Ticketing is used to manage issues and bugs.

Licence

Formerly the OGSA-DAI custom licence was used, now it's Apache 2.0.

NDG use BSD

Who provides advice on how licences work together? An OGSA-DAI licence FAQ was requested by users. OMII-UK should provide this advice also.

Conclusions

OGSA-DAI's proposed new repository structure seems sound. Evaluations of alternative tools for the repository, build and test scripting need to be done.

Further Work

OMII/OSSWatch - draw up guide on licence compatibilities for developers and publicise OSSWatch have advice on this OMII publicise this on their doc/WWW

Mike - Publicise OSSWatch advice on OGSA-DAI WWW site and in user doc

Mike - Find out how XP projects do system tests

Mike - Look at Maven

Mike - Look at Jython