Network Weather Service

Most space-sharing parallel computers presently operated by high-performance computing centers use batch-queuing systems to manage processor allocation. In many cases, users wishing to use these batch-queued resources have accounts at multiple sites and have the option of choosing at which site or sites to submit a parallel job. In such a situation, the amount of time a user’s job will wait in a batch queue can significantly affect the overall time a user waits from job submission to job completion.

Our batch queue prediction service (QBETS) offers queue delay predictions for individual jobs. Two types of predictions can be made:

• given the job characteristics, we can predict a statistical upper bound on how long the job is likely to spend waiting in the queue prior to execution

• given the job characteristics and a start deadline, we can calculate the probability that the job begins execution by the deadline

Predictions are available for more than a dozen supercomputing sites and new systems are being added constantly. (If you run a batch queue system and you would like us to generate predictions for it, please contact us.)

We have developed a number of interfaces to the QBETS service which we keep synchronized. We also maintain backwards compatibility by keeping old interfaces live until users have moved to the newest version (infrequent). The interfaces currently are:

• Website - our own custom web site
• C API - in the form of a standard portable C library
• UNIX Commandline tools - several frontend tools utilizing the C API
• Web service - including example Java and Perl client applications