Home Page

Introduction

Instrumentation

System & Database

Networking

Visualization

Science

Demonstrations

People

Publications

Related Projects

System and Database

• The Instrumentation Network consists of remotely deployed ``REINAS/PCs'' -- inexpensive Unix computers responsible for collecting, parsing, and in some cases processing data at the instrument, prior to reliably logging the data to local disk. Once logged, the REINAS architecture guarantees measurements will eventually reach the database and be available for queries. Instruments nodes within REINAS are Internet entities and use standard network protocols (TCP and UDP) for communicating with other parts of the system. Because instrument nodes are often deployed at remote sites with unreliable (often wireless) network connections, the node is typically capable of providing weeks or even months of reliable storage for measurements within the log; additional space can be easily added as necessary.

  The figure below details the software architecture at a REINAS instrument node collecting measurements from a meteorological station. Measurements arrive through a serial interface and are parsed by the met-distributor, which maintains direct control over the instrument and is the only software module containing instrument-specific code. Multiple metd daemons act as TCP rendezvous points for processes that wish to interact with the instrument data stream; metd also provides a generic instrument-independent interface for all meteorological and similar instruments. The met-distributor and metd construction allow multiple processes to receive real-time data and interact with the instrument through a generic, reliable, and secure interface.

• Each REINAS instrument stream begins with a logger, in this case the met-logger, which reliably writes each measurement record to a local First-In First-Out disk log. A generic reader is responsible for reading the logged measurements and forwarding them across potentially unreliable network connections to a database node [see below]. Only after the database node has logged the measurement will the reader delete the measurement from the instrument log and move on to the next record.

  Simultaneously with the migration of data from the instrument to the database, other processes may also monitor the real-time stream. Raw ``level zero'' instrument archives (termed local-products) may be created locally, or any number of interactive sessions may exist using standard TCP/IP tools such as telnet.

  ---

• Each Database Node consists of one or morestream-servers which log measurement data received from multiple remote readers to disk, as shown inthe figure below describing the database node software architecture.Only aftera measurement has been logged at the database node willthe corresponding reader delete it from the remote instrumentation node.

• For each log written by a stream-server, a correspondingdatabase loader process reads the log, inserting measurementsinto schema datatables within the database. Again, only aftereach insert has been confirmed will the loader remove theinserted measurement from the stream log.

  ---

• All Visualization and User Applications which requireaccess to data contained within a database node, communicatewith the database through a dispatcher which, except forcertain schema administration utilities, is the only processbesides the loader which directly interfaces with thedatabase. The dispatcher implements a low-leveldatabase and schema independent interface to the data containedwithin the database. It allows multiple independent andsimultaneous queries, which can be issued in parallel by separateuser level applications. Below is a sample real-time queryresult for stream 1014, a remote solar irradiance sensor.

  squery 1014211 id 1014214 tstamp 842394751 09/10/1996 22:32:31212 magnitude 703.359985212 uncertainty 0.000000195190
  

  For user applications that require it, the dispatchercan pass-through Standard Query Language (SQL) queriesto the database and return the result. Other application-level interfaces,such as the RSObjects API supporting visualization applications,have also been built ontop of the dispatcher.

  Note that it is possible for user-level applications to bypass thedatabase node and communicate directlywith an instrument node, such as themet-distributor in the example given earlier. This allowsfor remote real-time control and steering of instruments by scientists andother qualified users; in practice, however, a steering applicationwill communicate both with the database andinstrument node.