NSF’s vision of large-scale sensor-based Observing Systems, to address the National Research Council’s “Grand Challenges” for environmental science, relies on robust cyberinfrastructure. Streaming data middleware is an essential component of this cyberinfrastructure, providing the foundation for reliable data acquisition and instrument management services. The environmental science and engineering communities are currently engaged in the early planning and development phases of the next generation of large-scale sensor-based observing systems. The Open Source DataTurbine Initiative is a new NSF-funded project to empower these communities with Streaming Data Middleware.

The Open Source Data Turbine Initiative is sponsored by NSF award #OCI-0722067

Large-scale sensor-based observing systems face a number of significant challenges in design and operations, including heterogeneity of instrumentation and complexity of data stream processing. These systems incorporate instruments from across the spectrum of complexity, from temperature sensors to acoustic Doppler current profilers, to streaming video cameras. They typically require integration of instruments from various vendors and R&D labs. Managing these instruments and their data streams is a serious challenge. Requirements on the cyberinfrastructure include scalable and secure support for real-time data acquisition, instrument and data stream management, and analysis and visualization. Many applications address these issues by building custom systems that are inevitably complex and difficult to support. Others buy into a commercial proprietary system. Extensibility, scalability, and interoperability are often sacrificed under these approaches. To address these cyberinfrastructure challenges in a principled manner, we launched the Open Source DataTurbine Initiative.

DataTurbine is a robust open-source streaming data middleware system that satisfies the core requirements for sensor-based environmental observing systems. It is relatively mature and has been tested in a variety of real-world streaming data applications. It facilitates the development of complex distributed streaming data applications, including real-time virtual observatories and telepresence collaboratories. The Open Source DataTurbine Initiative combines commercial-quality software support with an active and engaged user and developer community. We invite you to share our vision and join the Open Source DataTurbine Initiative.

Open Source DataTurbine Overview -- e-Science 2007 Paper

The Open Source DataTurbine Initiative is sponsored by NSF award #OCI-0722067