Software Infrastructure for High-Performance Distributed Computing


Objective: The objective of this project is to develop a software infrastructure that enables the construction and programming of "networked virtual supercomputers." This computing environment couples high-speed networks and high-performance computers, allowing an application to exploit specialized collections of resources, even if those resources are geographically distributed.

Approach: Our approach to networked virtual supercomputing is based on a metacomputing infrastructure called Globus, and an object-oriented parallel and distributed programming language called Compositional C++ (CC++).

The Globus system defines a core set of services which are designed to support a range of different metacomputing environments. The core set of services includes communication, authentication, high-performance I/O and configuration management. Higher level services, implementing specific policies for scheduling, resource location, security, etc, are built on top of the core set of metacomputing services.

A variety of programming interfaces are defined as part of the high-level metacomputing services. One of these interfaces is the Compositional C++ (CC++) programming language. CC++ is based on C++, the most widely used sequential object-oriented programming language. CC++ consists of six small extensions to C++. Because CC++ is so similar to C++, a C++ programmer can easily learn to use CC++. The extensions were carefully designed to support to the expression of a wide range of parallel programming styles and to promote parallel library design and code reuse.

Accomplishments: We participated in the development of infrastructure and applications for the I-WAY, a large scale experimental testbed for high-performance distributed computing. We demonstrated a prototype application that performed real-time processing and interactive visualization of data obtained from a DMSP weather satellite data. This application is unique in that it was completely written in the CC++ programming language and mapped onto a prototype version of the Globus metacomputing infrastructure, deployed on the I-WAY.

Significance: Parallel virtual supercomputing represents an important new development and will have a profound impact on the NASA high-performance computing community. The ability to couple diverse, distributed, high-performance resources, such as large-scale databases, massively parallel supercomputers, and interactive visualization environments will enable entirely new classes of applications to be developed and deployed.

Status/Plans: The I-WAY is complete and the basic networking infrastructure has been dismantled. However, new high-speed networks, such as the vBNS are being deployed and we intend to exploit these environments. We are in the process of defining a testbed environment, called GUSTO, to further explore mechanisms for networked virtual supercomputing.

Compositional C++ has proven to be an effective tool in building high-performance distributed applications. We are extending these tools to run on new platforms that located at Caltech and JPL. These include the HPSS high performance I/O system, the Cray T3E and the soon to be acquired Convex SP-2000.

Point of Contact:
Dr. Paul Messina
Center for Advanced Computing Research
California Institute of Technology
(818) 395-3907

messina@cacr.caltech.edu

Dr. Carl Kesselman
Beckman Institute
California Institute of Technology
(818) 395-2756

carl@compbio.caltech.edu