SCIENCE TEAM III GRAND CHALLENGE INVESTIGATOR MILESTONES
High-Performance Cornerstone Technologies for the National Virtual Observatory
Thomas Prince, California Institute of Technology
| Num | Label | Milestone | Expected | Certified |
|---|---|---|---|---|
| 1. | A | Software Engineering Plan completed (MS Word, PDF) Publish the Project Management plan on a project web page. The plan will include the following: - Science use cases and detailed requirements, approved by the Customer Review Board (CRB). The CRB will be specifically asked to provide guidance on the spatial scales and pixel sampling frequencies over which science grade mosaics should be served. - Technical Approach. - Software Engineering Plan. - Management Approach. - Quality Assurance. - Configuration Management. Deliver a Risk Management strategy that will be available internally to the project and the sponsor. |
05/31/02 | 06/14/02 |
| 2. | E | Code baseline completed. Describe the physical problem that is addressed by the baseline code. The baseline code will include the ability to remove backgrounds through application of a low pass filter. - Baseline the performance of the yourSky image mosaic service, which offers custom access to mosaics of user-selected size, coordinate system, and projection, with bilinear sampling. The performance will be baselined on the JPL PowerOnyx. It is equipped with 8 x 200 MHz MIPS R10K processors and running at a clock speed of 194 MHz. Performance and scaling curves will be published on the project web page. Documented source code will be delivered to the project web page. - Background Removal Service - Baseline a general background removal service that provides background removal decoupled from the mosaic service. Perform a global fit to the 2MASS and DPOSS data with a high-pass filter that adjusts images to a common sky mode but preserves background flux. Write background parameters to a meta-database. Publish baseline performance to fit the images in the 47% of sky that is publicly released. Documented source code will be delivered to the project web page. All software referred to above will be documented and made publicly available through the project website. Publish on the project web page initial system requirements (MS Word, PDF) and design documents (MS Word, PDF). |
06/30/02 | 08/14/02 |
| 3. | H | Design policy for interoperability and community delivery agreed on. - Declare that this project will use Virtual Sky and OASIS as its primary clients. The API for other clients to run the mosaic engine will be published on the project web page. Declare that the operational version of the mosaic engine, yourSky, will run on the NPACI Teragrid, and that the project will serve mosaics from three surveys, 2MASS, DPOSS, and SDSS. Under the guidance of the Customer Release Board, we will establish policies for the public release of the service and for naming customers who will be permitted access to the service on a shared-risk basis before public release. Declare working relationships with collaborators at NPACI (SDSC), 2MASS, DPOSS, and SDSS. - The Design Policy for Interoperability and Community Delivery will be published on the project website. - Publish on the Project Web page updated requirements and design documents and initial test plans (MS Word, PDF). |
07/31/02 | 09/27/02 |
| 4. | B | First Annual Report delivered (MS Word, PDF). Submit FY02 Annual Report to CT via the Web. |
08/31/02 | 11/05/02 |
| 5. | F | First code improvement demonstrated. Develop Science Grade Mosaics that conserve energy and support background removal, with metrics specified through the guidance of the CRB and scientifically validated under its auspices. Access to this service will be through a modification of the existing YourSky web form. YourSky Mosaic Engine – - Ensure conservation of energy in mosaics. - Handle image rotations in all WCS projections. - Metric: The following metrics apply to science grade mosaics; their precise values will be established through the guidance of the CRB: - Reduction in the average deviation from the measured energy per unit area (we anticipate roughly 50%) when constructing mosaics in at least 10 WCS projections with any image rotation. - Spatial scale of mosaics and spatial re-sampling of pixels that allow science analysis (we anticipate 1 to 5 degrees spatial scale; and full, 1/2, 1/4, and 1/8 resolutions). - Apply Background Removal Parameters that support background subtraction models: - Common sky model that preserves total flux - Preserve point sources only - Preserves feature on a scale that allows science analysis(we anticipate 1 to 5 degrees, as noted above). Documented source code made publicly available via the project website. |
02/28/03 | 08/14/03 |
| 6. | 1 | Interoperability prototype. - The improved YourSky code delivered in Milestone F will run on the Teragrid Linux cluster. Performance comparison between the PowerOnyx and the Teragrid will be published on the web page. - The improved YourSky code delivered in Milestone F and running on the Teragrid will be interoperable with the OASIS and VirtualSky clients, in that users place an order for a custom mosaic through these clients, receive notification of the completion of the request, and are able visualize the images. |
07/30/03 | 02/04/04 |
| 7. | C | Second Annual Report delivered (MS Word, PDF). Submit FY03 Annual Report to CT via Web. |
08/30/03 | 02/04/04 |
| 8. | G | Second code improvement completed. - The improved YourSky code per milestone I will run on the Teragrid. The achievable computational speed-up will depend on the performance of the Teragrid as deployed. We propose two performance metrics: A target computation speedup that ignores I/O time and a target overall speed-up that includes both computation and I/O times. We will achieve a target performance that is equivalent to a computation speedup of 64 and an overall speedup, including I/O, of 32, for a 5 degree x 5 degree 2MASS mosaic (which will be the most computation intensive dataset) on a 128 x 1GHz (128 GFLOPS) target machine with a sustained bandwidth to disk of 160 MB/sec. - Cache results locally for commonly requested regions. Develop cache of at least 2 TB. Metric: Demonstrate speed-up when cached mosaic is requested. Publish speed-up figures (MS Word). Documented source code will be made publicly available via the project website. |
02/28/04 | 11/05/04 |
| 9. | J | Full interoperability demonstrated using improved codes. - Demonstrate that the compute engine accepts requests from the OASIS and VirtualSky clients for mosaics from the 2MASS, DPOSS, and SDSS surveys; processes the request (includes accessing cached images as necessary); and notifies the user regarding the status and availability of a mosaic, which can be visualized by the user. Visualization includes full user control of the image in real-time: pan/zoom, cropping, scaling, resampling, color table, stretch, and histogram equalization. - Publish on the project website updated requirements (MS Word, PDF) and design (MS Word, PDF) documents, updated test plan and test reports, and a draft User Guide. |
08/15/04 | 01/28/05 |
| 10. | K | Customer delivery accomplished. Community delivery of the fully operational service. Deployment will be announced at a major national astronomical conference, where we will: - Demonstrate custom mosaics from 2MASS, DPOSS, and SDSS with sustained throughput of 30 degrees squared (e.g. thirty 1 degree x 1 degree mosaics, one 5.4 degree x 5.4 degree mosaic, etc.) per minute on a 1,024 x 400Mhz R12K Processor Origin 3000 or machine equivalent with sustained bandwidth to disk of 160 MB/sec and all input data previously staged on these disks. - Present scaling curves specifying the variation in compute time with mosaic size and number of user requests. - Deliver a User Guide that will include descriptions of how to access the service, known caveats and limitations, and a thorough description of the impact of background removal on the scientific integrity of the images. Documented source code will be made publicly available via the project website. Publish on the project web page final versions of requirements, design, test plan and test results. Deliver to the project web page an updated User Guide and a Maintenance Guide. |
01/10/05 | 04/22/05 |
| 11. | D | Final Report delivered. Submit Final Report to CT via the Web. |
02/15/05 |
+ Home
