Efficient and Reliable Data Stream Management (PhD Thesis, finished)
The proliferation of sensor technology, especially in the context of embedded systems, and the progress of ubiquitous computing strongly supports new types of applications that make use of streams of continuously generated sensor data. Applications like telemonitoring in healthcare or roadside traffic management systems urgently require reliable data stream management (DSM) in a failure-prone distributed setting including resource-limited mobile and embedded devices. In this thesis, we present an overview of representative applications in the field of e-health and discuss applied research in the field. Moreover, we introduce a model for distributed DSM. On the basis of this model, we define levels of reliability and describe necessary consistency constraints for distributed DSM. In addition, we formally characterize possible failures that can occur at runtime and reliability levels we guarantee to applications. Finally, this model is also used as basis for an in-depth evaluation of reliability algorithms. In this thesis, we present different passive standby reliability algorithms based on operator checkpointing. In particular our proposed efficient coordinated operator checkpointing (ECOC) algorithm provides lossless and delay-limited reliable data stream management in an very efficient way. ECOC particularly focuses on critical applications, e.g. in healthcare, where the loss of data stream elements can not be tolerated. The ECOC approach considers fine-grained backups at operator level, which allows for the flexible and efficient usage of available resources in a network. Moreover, ECOC is optimized to reduce the overhead of checkpointing and to support complex stream process execution graphs, which include joins, splits and even cycles within data stream flows. The ECOC algorithm has been fully implemented in Java in OSIRIS-SE, our prototype of an information management infrastructure for distributed DSM and thus can even be run on mobile devices. Extensive evaluations within the OSIRIS-SE implementation proof the applicability of our approach and a demo prototype developed within the thesis gives hands-on experience on next generation e-Health applications.