Weather Data Warehouse: An Agent-Based Data Warehousing System

The quality and content of data regarding certain types of information can vary over time. Such data may be weather data, news reports, demographic data, and health data. To effectively utilize such data, it has to be gathered, processed, stored, and provided to information consumers in different formats. In case of different sources, the data must also be processed taking different formats and update frequencies into account. An agent-based system supports deploying and provisioning data processing services where they are best suited - at the information source, data warehouse, broker, or information consumer. We describe an agent-based approach to data warehousing enabling dynamic and seamless integration of new information sources and data formats. The system provides data to the ultimate information consumers on a subscription (push) or request (pull) basis. We have applied this architecture to provide a dynamic and flexible weather data warehouse that provides a wide variety of weather data from different sources to different weather-based applications.