# JOb Scheduling Hierarchically (JOSH) - Going Global with Globus V3 and Grid Engine

## Introduction

JOSH (JOb Scheduling Hierarchically) is a multi-site job-management tool built on top of the Globus Toolkit and Grid Engine. A JOSH user interface (or client) allows the submission, monitoring and termination of jobs running on Grid Engines at remote compute sites (the servers). JOSH also handles the transfer of data files and executables to and from jobs.

The JOSH software employs a hierarchical scheduler to match a user's job requirements against Grid Engine instances at available compute sites. It can then forward the job to the chosen compute site for local scheduling and execution.

A middleware layer handles secure communications and data transfer between the hierarchical scheduler and Grid Engine and any remote data sources. The OGSA-compliant Globus Toolkit version 3 forms this layer.

JOSH has been developed by the EPCC Sun Data and Compute Grids project. The project partners are the UK National e-Science Centre, represented in this project by EPCC, and Sun Microsystems. This project also developed the Transfer-queue Over Globus (TOG). TOG allows an enterprise to access remote compute resources via their existing Grid Engine installation.

## Background

Grid Engine allows the efficient use of compute resources within an organisation. However there is some desirable Grid functionality that Grid Engine does not provide. Grids can be classified at three different levels.

- Cluster Grid - a single team or project and their associated resources.
- Enterprise Grid - multiple teams and projects but within a single organisation, facilitating collaboration of resources across the enterprise.
- Global Grid - linked Enterprise and Cluster Grids, providing collaboration amongst organisations.

Grid Engine meets the first two levels by allowing a user to transparently make use of any number of compute resources within an organisation. However, Grid Engine, alone does not yet meet the third level.

The Globus Toolkit is essentially a Grid API for connecting distributed compute and instrument resources. Integration with Globus allows Grid Engine to meet this third level. That is, it allows collaboration amongst enterprises. The JOSH software integrates Grid Engine V5.3 and Globus Toolkit V3 to provide a user with this access to remote resources. This allows user jobs to be scheduled across a global grid.

## Download

The JOSH User Guide describes the use of the JOSH 1.1 command-line client tool known as hiersched. The JOSH Client Install Guide and the JOSH Server Install Guide describe the installation of the JOSH client and server software respectively. The Known Problems and Solutions document describes issues and limitations of JOSH 1.1, together with potential solutions and workarounds where appropriate. The Build Guide describes the build process for the JOSH software.

The JOSH software can be downloaded as a binary archive, or as a source archive:

- JOSH 1.1 Client Binary Archive: josh-1.1-client-bin.tar.gz
- JOSH 1.1 Server Binary Archive: josh-1.1-server-bin.tar.gz
- JOSH 1.1 Source Archive: josh-1.1-src.tar.gz
- JOSH 1.1 Javadoc Archive: josh-1.1-api.tar.gz

The client binary archive contains JOSH client binaries, the User Guide and the Client Install Guide. The server binary archive contains JOSH server binaries, the Server Install Guide and the JOSH limitations document. The source archive contains the JOSH source, build files, the Build Guide and all the other client, server and user documentation. The javadoc archive contains the Javadoc built from the source. The JOSH Functional Specification and System Design are available separately.

## Contacts

For more information on the project and its deliverables please access the Sun Data and Compute Grids project home page or contact the SunDCG project team.