Research on Reliability of Hadoop Distributed File System

With the development of cloud computing, more and more enterprises begin to develop their own cloud storage architecture by Hadoop. HDFS, at the bottom of Hadoop framework, stores files on all the nodes in cluster. This article first analyzed the HDFS architecture and put forward a problem single point of failure: There is only one NameNode in HDFS, which has limited the number of metadata storage and is prone to a failure of single node. Once the NameNode fault occurs, the system will break down. It affects the reliability of the cluster directly. Then it introduced Paxos algorithm and improved the structure of HDFS according to its principle. It solved the problem by increasing the number of NameNode and ensuring the synchronization of them by Paxos algorithm. At last, it verified the feasibility of the scheme by an experiment. A failure of metadata server will not interrupt the system so the reliability is improved.