Designing and evaluating hybrid storage for high performance cloud computing

The need for reliable and fast storage systems is increasingly critical in various fields including artificial intelligence and data analytics. This paper proposes a new architecture for large-scale data storage systems, focusing on comparing performance of software and hardware storage technologies that effectively reduce the computational latency and improve performance. The main contributions include: (1) the combination of Singled Magnetic Recording (SMR) for storing data and Solid State Devices (SSD) for storing metadata is a viable solution for implementing large data storage systems; and (2) the combination of Conventional Magnetic Recording (CMR) for storing data and SSD for storing metadata shows the highest performance for high performance computing. Our experiments are carried out in multiple settings, demonstrating that the proposed architecture successfully improves performance for sequential and random reads/writes. The prototypes are evaluated with a set of workloads, showing the superiority of the proposed data storage configurations. This work provides new opportunities for efficiently processing and storing data and metadata in large-scale data analysis systems.