Software Engineering Support of the Third Round of Scientific Grand Challenge Investigations

Task 4 Report - Earth System Modeling Framework Survey

Review of the Cactus Framework

By Bryan Talbot, Shujia Zhou, Glenn Higgins
bgtalbot@tasc.com, szhou@pop900.gsfc.nasa.gov, gjhiggins@tasc.com
Litton-TASC, 4801 Stonecroft Blvd, Chantilly, VA, 20151

10/04/2000

Introduction

Cactus [1] is a software framework resulting from the collaboration of numerous institutions including the Albert Einstein Institut [2], Washington University Gravity Group [3], National Center for Supercomputing Applications (NCSA) [4], Konrad-Zuse-Zentrum fuer Informationstechnik Berlin [5], International Relativity Group [6], Rechenzentrum Garching (RZG) der Max-Planck-Gesellschaft [7], Argonne National Laboratory [8], and Universitat de les Illes Balears [9].

As described on the Cactus web-page,

"Cactus is an open source problem solving environment designed for scientists and engineers. Its modular structure easily enables parallel computation across different architectures and collaborative code development between different groups. Cactus originated in the academic research community, where it was developed and used over many years by a large international collaboration of physicists and computational scientists.

"The name Cactus comes from the design of a central core (or "flesh") which connects to application modules (or "thorns") through an extensible interface. Thorns can implement custom developed scientific or engineering applications, such as computational fluid dynamics. Other thorns from a standard computational toolkit provide a range of computational capabilities, such as parallel I/O, data distribution, or checkpointing.

"Cactus runs on many architectures. Applications, developed on standard workstations or laptops, can be seamlessly run on clusters or supercomputers. Cactus provides easy access to many cutting edge software technologies being developed in the academic research community, including the Globus Metacomputing Toolkit, HDF5 parallel file I/O, the PETSc scientific library, adaptive mesh refinement, web interfaces, and advanced visualization tools." [1]

Synopsis

Prominent information about Cactus includes the following:

  1. Community of Origin: The Cactus framework originates in the relativistic physics community.
  2. Description: Cactus is a component-oriented application development environment for scientific computing which employs a common interface to components written in multiple languages.
  3. Team: The Cactus team includes at least twenty-five individuals, a partial list of which includes Gabrielle Allen [10], Tom Goodale [11], Ed Seidel [12], Thomas Radke [13], Gerd Lanfermann [14], and others ([15], p. 5). The team is actively functioning and is anxious to work with users from other communities to further enhance the code infrastructure.
  4. Maturity: As of August 2000, the Cactus team is shipping version 4.0, beta 8.0. Version 1.0 was released on April 24, 1997. Cactus spans at least three years and four major releases. The predecessors of Cactus go back to the early 1990's. Cactus benefitted from the NSF Black Hole Grand Challenge Project and several other attempts to develop a framework for research groups. Cactus has a long heritage in community framework development.
  5. Users: Cactus has approximately 100 users, mostly within the relativistic physics community, but recently expanding into other disciplines, such as aerodynamics, condensed matter, chemical engineering, geophysics, and climate modeling. Some of the users, as noted on the Cactus homepage, include the following:

    Within the relativistic physics community, application groups actively using Cactus include the following:

    Astrophysics users include the following:

    Aerospace users include:

    Condensed Matter Physics users include:

    Chemical Engineering groups include:

    Geophysics groups using Cactus include:

    Computational science groups that are actively using and/or developing Cactus include:

    Cactus, having a long history of Grid computing experiments, is becoming more extensively used in grid communities such as the following:

  6. Language: The core of Cactus is written in ANSI C, with PERL heavility used during the configuration process. There is no FORTRAN or C++ in the core code. Cactus currently accepts thorn modules in FORTRAN 77, FORTRAN 90, C, and C++. Support for other languages such as Java, PERL, and Python is planned in the next version release (4.1).
  7. Tools/Utilities Included: Cactus comes with several toolkits, mostly oriented toward relativistic physics. It also has a runtime system with a scheduler.
  8. Application Interface: Cactus employs a custom common interface to thorns (component modules) which is language independent and provides some object-oriented capabilities.
  9. Documentation: The Cactus team actively maintains a web-site [1], which provides links to a host of information. Printed documentation includes a users guide [39], quick start guide, and a set of developer tutorials [40] [41] . Additionally, the web-site provides access to numerous presentations [42] [43] [44] [45] [46] [47] [48] [49] [50] [51] [52] [53] [54] , publications about Cactus [55] [56] [57] [58] [59] , and application publications and posters [60] [61] [62] [63] [64] [65] [66] [67] [68] . Cactus has also been the focus of a workshop [69] [70] held at NCSA [4]. Thus, Cactus has a significant amount of associated documentation in its user community.
  10. Associated Software: Cactus requires PERL [71], Gnu Make [72] and a C/C++ compiler. It also interfaces with the following software:
  11. Performance: So long as most of the execution time is spent inside a module/thorn, performance of a climate application using Cactus should be about the same as before. The Cactus scheduler does not play a large role in the execution time. Cactus achieved 142 Gflops on a 1024 Cray T3E-1200 with a full thorn set solving the coupled Einstein-Hydro equations.
  12. Special Features:

Description

High-Level Design

Because of the high complexity of solving relativistic equations [58], described as "among the most complicated seen in mathematical physics", members of the relativistic physics community created Cactus as a common development tool which allowed multiple components, known as thorns, to be assembled and run in the same environment. The need for a modular system is driven by the desire to have members of the community work relatively independently on various software components. The need for multiple language support is derived from the fact that community members are developing components in various languages, including FORTRAN 77, FORTRAN 90, and C. Thus, these needs, in conjunction with other needs, translate into a need for a single development environment with a common interface that can span multiple language modules. The Cactus high-level design is governed by the following major ideas:

  1. "A framework is a reusable semi-complete application that can be specialized to produce custom applications." [89]

    Cactus is not just a tool-kit, a library, or a module that performs a particular function. It is a complete framework that meets the definition criteria, oriented towards the production of complete applications consisting of interacting objects.

  2. FORTRAN is a crucial scientific language that must be supported because speed is important.

    John Backus, a FORTRAN designer, was quoted in The Grid [90] [91] (p. 184) as saying the following with respect to the FORTRAN language.

    "It was our belief that if FORTRAN, during its first months, were to translate any reasonable "scientific" source program into an object program only half as fast as its hand-coded counterpart, then acceptance of our system would be in serious danger... To this day I believe that our emphasis on object program efficiency rather than on language design was basically correct. I believe that had we failed to produce efficient programs, the widespread use of languages like FORTRAN would have been seriously delayed." [92]

    For similar reasons, rather than abandoning FORTRAN and choosing an object-oriented language that is much more suitable to the design of their system, the Cactus team has put a great deal of effort into accommodating existing FORTRAN code.

  3. Modules should be self-contained with small interfaces where all communication takes place through the argument list.

    James Hack [93], in a discussion on modularization and coupling of climate models said the following:

    "Historically, scientific progress in atmospheric modeling has been slowed by technical difficulties of incorporating and testing physical parameterizations in different large-scale numerical models. Such problems are fundamentally linked to the fact that most codes are not modular in their design and often make use of very different data structures. Recognizing this problem, a number of scientists from major atmospheric modeling institutions have adopted a set of coding rules to make physics packages more "plug compatible" in order to facilitate their easy exchange [94]. Although this set of rules is specifically intended for physics packages that do not have a large number of their own prognostic variables, the conceptual approach is appropriate, and will ultimately prove necessary, for coupling complex climate system components (ocean circulation models, chemical models, biosphere models, etc.). Some of the more important concepts contained in this coding standard are:

    1. Each component should refer only to its own subprograms and a limited set of standard intrinsic functions.
    2. Each component should provide for its own initialization of static information and initial data through a single initialization entry point.
    3. All communication between packages shall take place though the argument list associated with a single unique entry point into each package.
    This approach to coupling major model components is quite reasonable." ([95], p. 317).

    Cactus works best with modules written in this style, where all communication with a module takes place through a single entry point using only parameters in the argument list. Modules using other means of communication or with multiple entry points will have more difficulty working with the Cactus system.

Cactus effectively integrates these three ideas at a high-level.

Use of the System

The enabling technology in Cactus is associated with the need for language-independent parameter and data passing. In Cactus, this is accomplished by having the user:

  1. Create special files in the module directory, independent from the source code, to declare and specify dominant parameters and data. These files provided a means to create data "objects" using inheritance and with public, private, and protected variables. Thorns may be scheduled in groups, with other routines/groups being scheduled before or after. A "schedule while (condition)" construct is also allowed to allow for some dynamic control and for loops. This is more primitive than a scripting language, except for dynamic change at runtime, but a scripting language is planned for the 4.1 release.
  2. Insert special Cactus keywords in the source code subroutine calling parameter lists instead of variable names.
  3. Compile the application using the Cactus system,

Cactus, in turn, builds the application by

  1. Scanning the special files to learn about the variables
  2. Inserting variable names automatically in the calling lists for each subroutine, and
  3. Compiling the final application using standard compilers.

When the application runs, Cactus performs the following functions

  1. Loads thorns
  2. Schedules functions from thorns. (Note that there is more than one entry point to a thorn. They are not treated as monoliths.)
  3. Transfers data between thorns
  4. Provides interfaces to visualization tools
  5. Provides interfaces to external web-browsers
  6. Writes data

The enabling technology, then is the Cactus source code, PERL [71] scripts, and interface conventions which enable it to accomplish these tasks. PERL is a language that is traditionally used for text processing because of its powerful pattern recognition capabilities. With Cactus, PERL and the C-preprocessor (CPP) are used as part of the build process but aren't part of the runtime. Cactus provides easy parallelisation for modules.

Views of the System

Cactus is a comprehensive system that can be described from several perspectives, each providing insight into what Cactus can do:

A Language Extension

In the sense that Cactus requires the user to insert special keywords in the source code, Cactus is a type of language extension. From this perspective, Cactus is similar in approach to High Performance FORTRAN ( [96] and [91], p. 188), an extension to FORTRAN in which areas of data parallelism can be denoted in the code via comments or several new keywords. Cactus is different from HPF in that the new keywords do not denote regions of parallelism but instead denote regions of the code where Cactus needs to substitute variable names at compile time. In a similar vein, whereas the principal focus of HPF is the layout of arrays ([91], p. 188), one of the main focuses of Cactus is storing array information in objects through an inheritance mechanism. However, Cactus is not an extension to a single language (such as FORTRAN), but is instead an extension to multiple languages (FORTRAN and C) using the same key-words. Because multiple languages are involved, Cactus also specifies unique names for types such as CCTK_REAL which corresponds to a real in FORTRAN or a double in C.

An Object-Oriented Language

In the sense that Cactus provides the means for users to build data structures in terms of other data structures using an inheritance mechanism it is a type of object-oriented language. Inheritance is a powerful feature of object-oriented languages and a key part of object-oriented frameworks ([89], p. 5). Inheritance is missing from both FORTRAN and C. Cactus currently provides this inheritance for data structures but not for method/function names, thus allowing "sub-classes" to be derived from previously defined data structures. Inheritance for function names will be available in the final release via function aliasing.

In addition to having inheritance, Cactus is like object-oriented languages in that thorns are similar to classes both in the manner of specification and in operation. For example, for a hypothetical MyClass class definition in the C++ language, the class variables are traditionally specified in two files: (1) MyClass.h, containing declarations, and (2) MyClass.C, containing member functions. Each member function of MyClass automatically has access to all data elements declared as part of the class in MyClass.h without passing in the function argument list. If the programmer decides that MyClass requires an additional data member, such as a gridded three-dimensional array, Z, then Z is added to MyClass.h and becomes automatically available to all member functions of MyClass. Thus the language provides the means to "pass" class variables to member functions without changing the function prototypes. In a similar fashion Cactus provides a means to declare "member data" in files separate from the thorn source code and automatically "pass" these to the thorn functions without requiring the user to change the function prototype.

Though Cactus is similar to object-oriented languages in form and function it is unlike them in elegance. Cactus is awkward in both form and function. Instead of a single declaration file (MyClass.h), three are required (interface.ccl, param.ccl, and schedule.ccl). Instead of meaningful type names like real or double, awkward names such as CCTK_REAL are used for Cactus-owned variables. Some of this awkwardness is unavoidable, a direct byproduct of trying to make something that interoperates between two dissimilar products. In the same way that a stereo system designed to operate in either a bus or a motorcycle, with two different mounting environments and two different acoustical environments would be awkward, the Cactus design is awkward because it straddles two different languages, thus requiring awkward conventions. Nevertheless, even the awkwardness can be a benefit in some cases, as CCTK_REAL can be defined as double or float, enabling users to easily configure the degree of precision in a computation.

A Compiler

In the sense that Cactus scans source code and looks for keywords, it is a type of compiler. Cactus uses GNU Make and PERL scripts to scan the source code, perform substitutions and arrange for the overall build of the application. It also relies on CPP, the C-preprocessor to perform macro substitutions. Because it calls the other compilers, it is a type of super-compiler that spans the multiple languages.

A Development Environment

In the sense that Cactus manages the production of the final application, it is a type of development environment. The user specifies code in "thorns" and Cactus provides assistance in assembling the components.

A Runtime Environment

In the sense that it provides scheduling of the various modules during program execution, it is a type of runtime environment. Cactus doesn't just compile source code written by somebody else, it also inserts specific code of its own to provide runtime scheduling between the different thorns. At a high level this is similar to the Parallel Application Workspace (PAWS) environment [97] where a central controller coordinates the running of different components. At a low level this is similar to a language like Java which generally runs on a virtual machine which creates, schedules, and controls the objects. This does not mean that there are two levels of control. It means that Cactus can be viewed in these two ways.

Evaluation

Cactus is unique among the frameworks we have examined because of its comprehensive modeling approach. It is worthy of close consideration by the climate community, not only for its implementation, but more importantly for its method . In many respects the needs of the climate community are not significantly different than the needs of the relativistic physics community or of any other community which develops and uses complex software and computer systems. The comprehensive manner in which Cactus addresses these universal needs provides valuable insights into what a framework should be and can become.

Cactus addresses the issue of how to promote the development of interchangeable software within a community by providing a tool with a comprehensive focus. The genius of the Cactus framework is not so much in the individual software elements as in the comprehensive nature of the solution, which enables the business of assembling and running an application to be effectively conducted. Individually, the elements may be either highly desirable to the climate community or, alternately, distasteful and awkward, But even some of the awkward elements, as part of a larger system, have a strength that other frameworks do not provide.

Strengths

In this light, these are the issues that the Cactus framework addresses:

  1. Cactus is a framework in the true sense. It addresses the need for a community software focal point by defining the nature of the software components and the relationships ( [89] p.4) between them and by providing a common tool to assist with assembly and scheduling.
  2. According to the pattern of true frameworks, Cactus provides a mechanism for inversion of control ( [89] p.5), a supervisory process (flesh) which schedules the interacting objects (thorns), thus relieving the module developer of that responsibility.
  3. Cactus addresses the reality of researchers working independently by developing the thorn/arrangement concept ([39], Chapter B1) which spans code, interface, and documentation. These concepts provide a common format to use that makes code more interchangeable and understandable among members of the community.
  4. Cactus addresses the reality of researchers working on a range of platforms because it is relatively platform independent, running on platforms ranging from laptops to supercomputers.
  5. Cactus simultaneous addresses two of the most difficult realities facing scientific computing today: It addresses these issues by defining a single interface which can be used with either language. The interface includes definitions of data type and array structures, and provides rudimentary inheritance capabilities which can be used with FORTRAN 77, FORTRAN 90, or C code. Thus, Cactus provides access to some object-oriented features without forcing users to migrate all of their code to an object-oriented language. This is a much better solution than defining an interface at the lowest level, corresponding to the intersection of languages.
  6. Cactus addresses the need for researchers to inspect a running process via the web [88]. Cactus provides utilities that allow web-browsers to connect to a running process for inspection. The Cactus homepage [1] provides a sample connection. It is important to note that it is not only possible to inspect via a browser, but that it is also possible to retrieve any data from the memory of a running simulation and have it streamed in HDF5 format to a local host, for more indepth visualization and analysis.
  7. Cactus has a strong role in the emerging area of Grid computing because it enables applications, which are not Grid aware, to run in a Grid environment, thus saving the researchers the effort of building-in this ability.

Weaknesses

Cactus has the following weaknesses.

  1. The standard Cactus arrangements [98] include little code that is directly applicable to the climate community because they are oriented towards Cartesian grids instead of geographical grids. Current arrangements include: Therefore, the climate community would have to develop/adapt their own code and modules. Nevertheless, as demonstrated by the spectrum of users, the Cactus team is anxious to work with communities like the climate community to extend Cactus functionality in specific ways.
  2. The scheduler may not be as flexible as desired for some climate simulations. Some climate simulations run models sequentially with different numbers of iterations for each model with intermediate coupling exchanges. This can be accommodated to some degree using the WHILE functionality. Nevertheless, having a real scripting language would provide more powerful capabilities.
  3. The interface common interface between FORTRAN/C is awkward. By moving the subroutine variables out of the subroutine into separate files it becomes harder to see what is being passed in and out. Nevertheless, hidden arguments are only those associated with the scheduler. All other arguments can be declared in the code as before.
  4. Saying that CACTUS works with FORTRAN, C, etc. is slightly misleading. Once a code is converted to CACTUS style it is committed and can be used on no other platform. If the Cactus-modified interface subroutine is small, then all of the lower-level subroutines can be pure FORTRAN or C so this isn't a problem. At the same time, however, this limits the benefits of data "objects" because they have to be immediately decomposed into variables and passed around. On the other hand, if the choice is made to pass around data as Cactus "objects" at all levels, then the code is fully Cactus-ized and will not compile or work on any other platform. Thus, when a community chooses Cactus, it must do so with a full commitment. This is a weakness only in cases where interchanging code with non-Cactus communities is important or where Cactus cannot be a complete solution for the climate community. Where such an exchange is necessary, one way to get around this problem is to define a macro to refer to CCTK_ARGUMENTS when using cactus and to refer to the actual list of parameters when not using Cactus.
  5. Cactus does not have a scripting language, as does ROOT [99], a capability that could be helpful. This capability is planned for release 4.1
  6. The type of inheritance supplied by Cactus is not the same type of inheritance that is integrated into object-oriented programming languages. Therefore, it has a limited ability to solve the types of problems that inheritance is typically used for.

Summary

Cactus is a complete framework solution to scientific modeling. It addresses the issue of differences between FORTRAN and C with a common interface and provides a development environment and a runtime environment, including a scheduler, for the modules. It also provides a means to inspect running code. It has a large amount of documentation, is used by many organizations, is stable, and has a supportive development team. The main disadvantage with Cactus is that it may require some adaptation before it can be used by the climate community. Nevertheless, the Cactus team is highly motivated to work with the climate community and other communities to extend Cactus capabilities.


References

1
CACTUS (CACTUS). URL: http://www.cactuscode.org

2
Albert Einstein Institute (AEI). URL: http://www.aei-potsdam.mpg.de

3
Washington University Gravity Group (WUGRAV). URL: http://wugrav.wustl.edu/

4
National Center for Supercomputing Applications (NCSA). URL: http://www.ncsa.uiuc.edu/newtest/NCSA/whatisncsa.html

5
Konrad-Zuse-Zentrum fur Informationstechnik Berlin (ZIB). URL: http://www.zib.de

6
International Numerical Relativity Group (INRG). URL: http://jean-luc.ncsa.uiuc.edu

7
Rechenzentrum Garching der Max-Planck-Gesellschaft (RZG). URL: http://www.rzg.mpg.de

8
Argonne National Laboratory (ANL). URL: http://www.anl.gov

9
Universitat de les Illes Balears (UIB). URL: http://www.uib.es

10
Gabrielle Allen, allen@aei-potsdam.mpg.de.

11
Tom Goodale, goodale@aei-potsdam.mpg.de.

12
Ed Seidel, +49 331 567-7210, eseidel@aei-potsdam.mpg.de. URL: http://jean-luc.ncsa.uiuc.edu/People/Ed

13
Thomas Radke, tradke@aei.mpg.de.

14
Gerd Lanfermann, lanfer@aei.mpg.de.

15
Masso, Joan, 1999: Cactus Philosophy, Cactus Workshop, NCSA, Sep. URL: http://www.cactuscode.org/Workshops/NCSA99/talk2/index.htm

16
AGAVE Code Project (AGAVE). URL: http://www.astro.psu.edu/users/nr/Agave

17
Michael L. Norman, norman@ncsa.uiuc.edu. URL: http://www.astro.uiuc.edu/department/faculty/norman.html

18
EU Network Project (EU_NetworkProject). URL: http://www.aei-potsdam.mpg.de/research/astro/eu_network/description.html }

19
NASA Neutron Star Grand Challenge Project (NASA_NeutronStarGrandChallenge). URL: http://wugrav.wustl.edu/Relativ/nsgc.html

20
Astrophysics Simulation Collaboratory (ASC). URL: http://wugrav.wustl.edu/ASC/mainFrame.html

21
Staff, ASC, 2000: NSF Progress Report, Astrophysics Simulation Collaboratory. URL: http://wugrav.wustl.edu/ASC/report1.html

22
Deutsche Luft- und Raumfahrtzentrum (DLR). URL: http://www.dlr.de

23
James Sethna, 607-255-5132, sethna@lassp.cornell.edu. URL: http://www.lassp.cornell.edu/sethna/sethna.html

24
Kenneth Bishop, 785-864-2918, kbishop@ukans.edu. URL: http://www.engr.ukans.edu/cpe-grad/bishop.html

25
William Bosl, 650-725-5835, bosl@pangea.stanford.edu. URL: http://pangea.stanford.edu/~bosl

26
Digital Earth Project (Digital Earth). URL: http://pangea.stanford.edu/~bosl/EarthObjectsPage/EarthObsPage.htm

27
National Energy Research Scientific Computing Center (NERSC). URL: http://hpcf.nersc.gov/

28
Michael Holst, 858-534-4899, mholst@math.ucsd.edu. URL: http://www.scicomp.ucsd.edu/~mholst

29
Charlie Crabb, 925-424-3265, ccrabb@llnl.gov.

30
Lawrence Livermore National Laboratory (LLNL). URL: http://www.llnl.gov

31
Benger, Werner, 2000: Porting CCTK to NT at the Cornell Velocity NT Cluster. URL: http://www.cactuscode.org/Presentations/Cornell2_August00.ppt

32
Cornell Theory Center (CTC). URL: http://www.tc.cornell.edu/ctc.html

33
European Grid Project (EGRID). URL: http://www.egrid.org

34
US Grid Project (GridForum). URL: http://www.gridforum.org

35
Grid Application Development Software Project (GrADS). URL: http://hipersoft.cs.rice.edu/grads

36
Ken Kennedy. URL: http://www.cs.rice.edu/~ken

37
GLOBUS (GLOBUS). URL: http://www.globus.org

38
German Gigabit Testbed Project (ZIB_TIKSL). URL: http://www.zib.de./Visual/projects/TIKSL/

39
Staff, Cactus, 2000: Cactus 4.0 Users' Guide. URL: http://www.cactuscode.org/Documentation/UsersGuide_html/UsersGuide.html

40
Team, The Cactus, 2000: Cactus 4.0 A First Tutorial: The 3D Scalar Wave Equation. URL: http://www.cactuscode.org/Tutorial/sld001.htm

41
Allen, Gabrielle, 1999: A First Tutorial of Cactus 4.0 - The 3D Scalar Wave Equation. URL: http://www.cactuscode.org/Presentations/Tutorial/index.html

42
Allen, Gabrielle, 2000: The Cactus Code: A Parallel Collaborative Framework for Large Scale Computing. URL: http://www.cactuscode.org/Presentations/SciComp_August00.sdd

43
Allen, Gabrielle, 2000: The Cactus Code: a Problem Solving Environment for the Grid, HPDC 9, Pittsburgh, Aug. URL: http://www.cactuscode.org/Presentations/HPDC9_August00.ppt

44
Dramlitsch, Thomas, 2000: Exploring Distributed Computing Techniques with Cactus and Globus, Globus Retreat, Aug. URL: http://www.cactuscode.org/Presentations/GlobusRetreat_August00.ppt

45
Seidel, Ed, 2000: Cactus/TIKSL/KDI/Portal Synch Day, Cactus Discussion Day, NCSA, July 28. URL: http://www.cactuscode.org/Presentations/NCSA_July00.ppt

46
Seidel, Ed, 2000: Cactus in GrADs. URL: http://www.cactuscode.org/Presentations/GrADs_July00.ppt

47
Lanfermann, Gerd, 2000: Cactus in a Nutshell, Sun HPC Consortium, Manheim, Jun. URL: http://www.cactuscode.org/Presentations/SUNHPC_June2000.ppt

48
Goodale, Tom, 2000: Cactus - The Future, Supercomputing 2000. URL: http://www.cactuscode.org/Presentations/Mannheim_June2000.sdd

49
Allen, Gabrielle, 2000: The Cactus Code: A Framework for Parallel Computing, Conundrum Series, Lawrence Berkeley Laboratory, May. URL: http://www.cactuscode.org/Presentations/LBL_May00.ppt

50
Goodale, Tom, 2000: The Cactus Code for Numerical Relativity. URL: http://www.cactuscode.org/Presentations/Garching_March00.sdd

51
Goodale, Tom, 2000: Cactus Computational Toolkit. URL: http://www.cactuscode.org/Presentations/Southampton_February2000.sdd

52
Goodale, Tom, 1999: The Cactus Computational Toolkit. URL: http://www.cactuscode.org/Presentations/FB_September99.sdd

53
Allen, Gabrielle, 1999: Cactus 4.0, HPDC8, Redondo Beach, Aug. URL: http://www.cactuscode.org/Presentations/HPDC8_August99.ppt

54
Allen, Gabrielle, 1999: MetaComputing Within the Cactus Framework, Globus Retreat, Rodondo Beach, Aug. URL: http://www.cactuscode.org/Presentations/GlobusRetreat_August99.ppt

55
Allen, Gabrielle, Thomas Dramlitsch, Ian Foster, Tom Goodale, Nick Karonis, Matei Ripeanu, Ed Seidel and Brian Toonen, 2000: Cactus-G Toolkit: Supporting Efficient Execution in Heterogeneous Distributed Computing Environments. , . URL: http://www.cactuscode.org/GordonBell_2000.ps.gz

56
Allen, Gabrielle, Werner Benger, Tom Goodale, Hans-Christian Hege, Gerd Lanfermann, Andre Merzky, Thomas Radke and Edward Seidel, 2000: The Cactus Code: A Problem Solving Environment for the Grid, Proceedings of HPDC 9, Pittsburg, Jun. URL: http://www.cactuscode.org/HPDC9_2000.ps.gz

57
Allen, Gabrielle, Tom Goodale, Gerd Lanfermann, Thomas Radke and Edward Seidel, 2000: The Cactus Code: A Problem Solving Environment for the Grid, Proceedings of First Egrid Meeting, Poznan, Mar. URL: http://www.cactuscode.org/CacPapers/Egrid_2000.ps.gz

58
Allen, Gabrielle, Tom Goodale, Gerd Lanfermann, Thomas Radke, Edward Seidel, Werner Benger, Hans-Christian Hege, Andre Merzky, Johan Masso and John Shalf, 1999: Solving Einstein's Equations on Supercomputers. IEEE Computer, 32, 52-59. URL: http://www.computer.org/computer/articles/einstein_1299_1.htm

59
Allen, Gabrielle, Tom Goodale, Joan Masso and Edward Seidel, 1999: The Cactus Computational Toolkit and Using Distributed Computing to Collide Neutron Stars, Proceedings of the Eighth IEEE International Symposium on High Performance Distributed Computing, Aug, 57-61. URL: http://www.cactuscode.org/CacPapers/HPDC8_1999.ps.gz

60
Seidel, Edward and Wai-Mo Suen, 1999: Numerical Relativity as a Tool for Computational Astrophysics. JCAM, . URL: http://xxx.lanl.gov/abs/gr-qc/199904

61
Alcubierre, Miguel, Gabrielle Allen, Bernd Bruegmann, Gerd Lanfermann, Edward Seidel, Wai-Mo Suen and Malcolm Tobias, 2000: Gravitational Collapse of Gravitational Waves in 3D Numerical Relativity. Phs. Rev., D61. URL: http://xxx.lanl.gov/abs/gr-qc/9904013

62
Arbona, A., C. Bona, J. Masso and J. Stela, 1999: Robust Evolution System for Numerical Relativity. Phs. Rev., D60. URL: http://xxx.lanl.gov/abs/gr-qc/9902053

63
Alcubierre, M., S. Brandt, B. Bruegmann, C. Gundlach, J. Masso, E. Seidel and P. Walker, 2000: Test-beds and Applications for Apparent Horizon Finders in Numerical Relativity. Class. Quant. Grav., 17, 2159-2190. URL: http://xxx.lanl.gov/abs/gr-qc/9809004/

64
Font, J.A., M. Miller, W. Suen and M. Tobias, 2000: Three Dimensional Numerical General Relativistic Hydrodynamics: I Formulations. Phs. Rev., D61. URL: http:xxx.lanl.gov/abs/gr-qc/9811015

65
Bona, Carlos, Joan Masso, Edward Seidel and Paul Walker, 1998: Three Dimensional Numerical Relativity with a Hyperbolic Formulation, . URL: http://xxx.lanl.gov/abs/gr-qc/9804052/

66
Walker, Paul, 1997: Three Dimensional Numerical Relativity with a Hyperbolic Formulation. URL: http://www.cactuscode.org/CacPapers/cactus1.ps.gz

67
Alcubierre, Miguel, Carsten Gundlach and Florian Siebel, 1996: Using Geodesics to Compare Exact and Numerical Spacetimes. URL: http://www.cactuscode.rog/CacPapers/MA_GR15.ps.gz

68
Alcubierre, Miguel, 1997: Evolution of Brill Waves in 3D Progress Report, GR15 Meeting, Puna/India. URL: http://www.cactuscode.org/CacPapers/BRILL_GR15.ps.gz

69
High Performance Computing and Cactus Computational Toolkit: A Framework for Solving PDEs in Computational Science with Special Advanced Topics in Astrophysics and Relativity, 1999: , NCSA, Sep. URL: http://www.ncsa.uiuc.edu//SCD/Training/CactusAgenda.html

70
Cactus Day at NCSA, 1999: , NCSA, Sep. URL: http://www.cactuscode.org/Workshops/NCSA99/index.html

71
PERL (PERL). URL: http://www.perl.org

72
GNU (GNU). URL: http://www.gnu.org

73
Concurrent Versions System (CVS). URL: http://www.cyclic.com

74
GNU Utilities for Windows (Cygwin). URL: http://sourceware.cygnus.com/cygwin/

75
Message Passing Interface (MPI). URL: http://www-unix.mcs.anl.gov/mpi/

76
GLOBUS (GLOBUS). URL: http://www.globus.org

77
FlexIO (FlexIO). URL: http://zeus.ncsa.uiuc.edu/~jshalf/FlexIO/

78
Hierarchical Data Format Version 5 (HDF5). URL: http://hdf.ncsa.uiuc.edu/whatishdf5.html

79
IEEE IO (IEEEIO). URL: http://zeus.ncsa.uiuc.edu/~jshalf/FlexIO/IEEEIO.html

80
LCA Vision (LCAVision). URL: http://zeus.ncsa.uiuc.edu/~miksa/LCAVision.html

81
Amira (Amira). URL: http://amira.zib.de

82
Portable Extensible Toolkit for Scientific Computing (PETSC). URL: http://www.mcs.anl.gov/petsc/

83
Grid Adaptive Computational Engine (GrACE). URL: http://www.caip.rutgers.edu/~parashar/TASSL/Projects/GrACE/index.html

84
Autopilot (Autopilot). URL: http://www-pablo.cs.uiuc.edu/Project/Autopilot/AutopilotOverview.htm

85
OpenDx (OpenDx). URL: http://www.opendx.org

86
Performance Data Standard and API (PAPI). URL: http://icl.cs.utk.edu/projects/papi/index.html

87
Scalalable Parallel I/O System (Panda). URL: http://cdr.cs.uiuc.edu/panda/

88
Benger, Werner, 1999: Web Cactus, NCSA Cactus Workshop, Sept 27. URL: http://www.cactuscode.org/Workshops/NCSA99/talk19/index.htm

89
Fayad, Mohamed E., Douglas C. Schmidt and Ralph E. Johnson, 1999: Building Application Frameworks. Wiley. URL: http://www.amazon.com/exec/obidos/ASIN/0471248754/qid%3D968778251/102-0715276-9734544

90
Foster, Ian and Carl Kesselman (Ed.), 1999: The Grid: Blueprint for a New Computing Infrastructure. Morgan Kaufmann Publishers, Inc.. URL: http://www.mkp.com/books_catalog/catalog.asp?ISBN=1-55860-475-8

91
Kennedy, Ken, 1999: Compilers, Languages and Libraries, In The Grid, Ian Foster and Carl Kesselman (Ed.), Morgan Kaufman Publishers, Inc., 181-204.

92
Backus, J., 1978: The History of FORTRAN I, II, and III. ACM SIGPLAN Notices, 13, 165-180.

93
James Hack, 303-497-1387, jhack@ucar.edu. URL: http://www.cgd.ucar.edu/asr98/cms.html

94
Kalnay, E., M. Kanamitsu, J. Pfaendtner, J. Sela, M. Suarez, J. Stackpole, J. Tuccillo, L. Umscheid and D. Williamson, 1989: Rules for the Interchange of Physical Parameterizations. Bull. Am. Met. Soc., 70, 620-622.

95
Hack, James J., 1995: Climate System Simulation: Basic Numerical and Computational Concepts, In Climate System Modeling, Kevin E. Trenberth (Ed.), Cambridge University Press, 283-318.

96
Koelbel, Charles, 1996: High Performance Fortran in Practice. URL: http://www.cs.rice.edu/~chk/hpf-tutorial.html

97
Parallel Application Work Space (PAWS). URL: http://www.acl.lanl.gov/paws/

98
Lanfermann, Gerd, 1999: The Standard Cactus Arrangements, NCSA Cactus Workshop, Sept 27.

99
ROOT (ROOT). URL: http://root.cern.ch/Welcome.html