Network Weather Service

The Network Weather Service is a distributed system that periodically monitors and dynamically forecasts the performance various network and computational resources can deliver over a given time interval. The service operates a distributed set of performance sensors (network monitors, CPU monitors, etc.) from which it gathers readings of the instantaneous conditions. It then uses numerical models to generate forecasts of what the conditions will be for a given time frame. We think of this functionality as being analogous to weather forecasting, and as such, the system inherits its name.

We have developed the NWS for use by dynamic schedulers and to provide statistical Quality-of-Service readings in a networked computational environment. The AppLeS scheduling methodology makes extensive use of its facilities and we have developed prototype implementations for Globus and the Global Grid Forum (GGF) Grid Information System (GIS) architecture.

Currently, the system includes sensors for end-to-end TCP/IP performance (bandwidth and latency), available CPU percentage, and available non-paged memory. The sensor interface, however, allows new internal sensors to be configured into the system. This particular interface is not what it should be -- we are working on it.

The current set of supported forecasting methods treat successive measurements from each monitor as a time series. Our initial methods fall into three categories:

• mean-based methods that use some estimate of the sample mean as a forecast,
• median-based methods that use a median estimator, and
• autoregressive methods.

The system tracks the accuracy (using prediction error as an accuracy measure) of all predictors, and uses the one exhibiting the lowest cumulative error measure at any given moment to generate a forecast. In this way, the NWS automatically identifies the best forecasting technique for any given resource. Moreover, as new methods are added, they will automatically be used to forecast the resource performance for which they are the most accurate.