Transfer of ESIS to Scientific Institutes

S.G. Ansari & P. Donzelli

ESA Information Systems Division, ESRIN, Frascati, Italy

E. Simonsen

CRI A/S, Space Division, Denmark

F. Ochsenbein

Centre de Données astronomiques de Strasbourg, France

In the year leading up to the transfer, the system was adapted to support the institutes' environments. In order to fulfil all requirements and to establish a tool in which the scientific community can cooperate and access several archive sites, a multi-server/multi-client architecture, based on the World Wide Web, was adopted.

Introduction

The European Space Information System (ESIS) provides 'homogeneous access to heterogeneously and geographically distributed data archives' in the astronomical and space physics domains. It was initiated in 1987 as a pilot project to support the space science community, but has evolved over the years to adapt to the changing needs and requirements of the community.

It allows a uniform access, retrieval, visualisation and manipulation of archived data. The main tools of the system are based on a Catalogue Browser, an Imaging package, a Spectral package and a Timing package. With the help of the SIMBAD database, located at the Centre de Données astronomiques de Strasbourg in France, used as a celestial name and coordinates resolver, ESIS was the first astronomical system to provide this resolver feature to the astronomical community to allow the user to access multiple astronomical catalogues.

ESIS, however, had to be transferred to several European astronomical institutes that had expressed interest in obtaining one of the components. The system therefore had to be adapted to support the different environments that the various institutes operate. Prior to the final transfer in December 1995, all ESIS applications were ported to support access to data through the World Wide Web. Not only was the Catalogue Browser with its form- based interface adapted to World Wide Web browsers, such as Netscape and NCSA Mosaic, but all ESIS applications now act as World Wide Web clients to access the various archived data worldwide.

ESIS has two components: the astronomical ESIS and the space physics ESIS. The astronomical system is discussed here. The space physics system transferred to the scientific institutes is similar in concept.

Organisation of astronomical observations

Data from astronomical observations are basically organised in two different ways: data- bases containing catalogues, and archives containing data sets (Fig. 1). The catalogues and data archives are distributed worldwide.

Figure 1. Organisation of astronomical data. User tools must be able to access astronomical catalogue and archives of astronomical datasets.

A catalogue database consists of tables representing astronomical catalogues and mission logs. An astronomical catalogue is a table of astronomical objects and associated measurements. The measurements are expressed in a variety of physical units, depending on the wavelength region in which they are observed. A mission log is a list of astronomical observations for a single mission or instrument. Entries in mission logs refer, most often, to one or more data sets for a particular mission, located in different data archives.

The principal categories of astronomical datasets are images, spectra and light-curves,

Purpose of ESIS

ESIS has three main functions:

• To provide a search facility, which can be used to search multiple databases of astronomical catalogues, with the databases being geographically distributed and heterogeneous.

• To provide a facility for on-line retrieval of astronomical datasets, integrated with the search facility.

• To provide a set of astronomical data visualisation and manipulation packages, integrated with the search and data retrieval facilities.

The multi-client/multi-server problem

A problem arose, however, with the transfer of the components to the institutes because the different sites have different Database Management Systems (DBMSs), rendering access to the astronomical catalogues and archived datasets incoherent.

Some sites use the Sybase Database System, while others use Oracle, and NASA/HEASARC has developed its own database system called Browse. In order to provide transparent access to the DBMSs and catalogue interoperability, catalogue servers must - independently of their specific DBMS - appear the same towards the clients. The interoperability between catalogue servers has been achieved through the introduction of an intermediate interface layer, referred to as the Uniform Query Engine (UQE), based on two principles:

1. A clear definition of all necessary interactions required by the system to support the basic access and retrieval functions (Fig. 2). This is achieved by implementing DBMS-specific query engines. In the case of Sybase and Oracle, the uniform layer is based on the Space Telescope Database (STDB) library. In the case of Browse, the HEASARC Database (HDB) library is used.

2. A Reference Directory containing all the necessary meta data, describing catalogues and their physical parameters, to uniformly access a remote site.

Figure 2. Overview of the ESIS architecture, showing the interactions between software components. An intermedie interface layer, the Uniform Query Engine (UQE), was introducetd to allow interoperability between clients and catalogue servers.

Astronomical datasets are made available from archives by means of three different protocols: File Transfer Protocol (FTP), Hypertext Transport Protocol (HTTP), and client/server-specific tools based on the native DBMS. For example, to de-archive preview HST data, a client/server provided by the Canadian Astronomical Data Centre had to be integrated with ESIS applications.

The Catalogue Browser

The ESIS WWW Catalogue Browser (WCB) allows direct access to astronomical catalogues at any archive site supporting a WWW server with ESIS gateway scripts (Fig. 3).

Figure 3. Examples of the WWW Catalogue Browser accessible from the CDS. On the top, the first screen from which a user selects catalogues to make subsequent searches. On the bottom, a typical result from the Hipparcos Input Catalogue.

This component of ESIS has been transferred to the Centre de Données astronomiques de Strasbourg (CDS) in France and is now provided as the 'VizieR service'. Being the astronomical data hub for the world, the CDS provides astronomical catalogues and an increasingly large amount of published tables of the Astronomy and Astrophysics journal. As part of the CDS overall services, the VizieR Catalogue Browser plays a major role in allowing the scientific community to search, browse and retrieve catalogued entries. There are currently almost 800 catalogues in the VizieR service.

ESIS applications

The ESIS visualisation and manipulation packages are Graphical User Interfaces that allow the search and retrieval of datasets from any remote archive site running a World Wide Web server. All three types of datasets are supported: images, spectra, and light curves.

Based on the Xanadu library, which was originally developed by the Exosat mission scientific team, each package contains its own search facilities to browse through multi-wavelength data using the standard search paradigms provided for catalogue browsing: by coordinate or by parameter.

Once a dataset is identified, it is de-archived and downloaded to the user's platform.

All searches in ESIS applications are done transparently, without the user having to first identify the location of an archive site. The applications are supplied with a configuration file that allows users to declare all archive sites running a World Wide Web server with the set of ESIS-specific access scripts (Figure 4 shows an example of a spectral application).

Figure 4. An example of the ESIS spectral application: the search menu (top left) and a plot of the spectral data

The ESIS applications are now being transferred to four institutes: the European Southern Observatory in Germany; the SAX mission group at the Italian Space Agency; the Brera Observatory in Italy; and the Monte Porzio Observatory, also in Italy.

Conclusions

The latest version of ESIS (Version 4), which was transferred to the scientific institutes, has provided a solution to the problem of transparent access to multiple and heterogeneous catalogues located on distributed query servers under different DBMSs. The feasibility of the adopted concepts and the implemented solution has been confirmed during the operational phase of the CDS/VizieR catalogue service throughout the last few months.

A few rather pragmatic choices were made with respect to the system architecture in order to achieve an operational and stable system with the limited resources available for the development:

1. Instead of applying a free and general query language, the ESIS Catalogue Browser has prioritised simplification through the definition of an essential set of client/server interactions. So- called field-constraints have been used in order to refine queries. Although restricted, the set of UQE interactions has allowed client software to be developed with sufficient functionalities.

2. The extensive use of meta data about catalogues and fields has allowed the transparent access to heterogeneous catalogues. With this approach, complexity due to data heterogeneity was moved from software coding to the creation and management of the Reference Directory. In addition, the software code has been made independent of any specific service, largely reducing the software and database maintenance tasks.

3. Choosing to keep the UQE stateless has contributed to the system's low complexity and more modular software components. The overhead of state transfers between client and server has not affected system performance.

The evolution of ESIS over the past year has been driven by several principles:

1. The existence of a system, well-defined in terms of user requirements and context.

2. Adaptation of existing archive-specific components to the overall system, thereby ensuring compatibility at all times.

3. Early involvement of the institutes that will provide the service with the transferred software, which allowed them to keep abreast of the status of the development process through each phase of the project. This guaranteed collaboration at the technical level throughout the development period and ensured that the software received was fully compatible with the target environment at the receiving institutes.

Acknowledgements

The authors wish to thank the Archive groups at the European Southern Observatory, the Canadian Astronomical Data Centre, and the HEASARC/ NASA for their technical assistance provided throughout the development period. Vitrociset and Cap Gemini were involved through industrial contracts.