The critical care data exchange format: a proposed flexible data standard for combining clinical and high-frequency physiologic data in critical care

Objective To develop a standardized format for exchanging clinical and physiologic data generated in the intensive care unit. Our goal was to develop a format that would accommodate the data collection pipelines of various sites but would not require dataset-specific schemas or ad-hoc tools for decoding and analysis. Approach A number of centers had independently developed solutions for storing clinical and physiologic data using Hierarchical Data Format - Version 5 (HDF5), a well-supported standard already in use in multiple other fields. These individual solutions involved design choices that made the data difficult to share despite the underlying common framework. A collaborative process was used to form the basis of a proposed standard that would allow for interoperability and data sharing with common analysis tools. Main results We developed the HDF5-based Critical Care Data Exchange Format (CCDEF) which stores multiparametric data in an efficient, self-describing, hierarchical structure and supports real-time streaming and compression. In addition to cardiorespiratory and laboratory data, the format can, in future, accommodate other large datasets such as imaging and genomics. We demonstated the feasibility of a standardized format by converting data from three sites as well as the MIMIC III dataset. Significance Individual approaches to the storage of multiparametric clinical data are proliferating, representing both a duplication of effort and a missed opportunity for collaboration. Adoption of a standardized format for clinical data exchange will enable the development of a digital biobank, facilitate the external validation of machine learning models and be a powerful tool for sharing multiparametric, high frequency patient level data in multisite clinical trials.